Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SDK-222] [no connection serial ] + implement missing internal presence map + presencequeue #621

Merged
merged 125 commits into from
Dec 13, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
125 commits
Select commit Hold shift + click to select a range
c950d26
Refactored realtime presence for loop, added spec annotation
sacOO7 Aug 14, 2023
60974f8
Added internal members property to realtime presence struct
sacOO7 Aug 14, 2023
6aa2873
Added synthesized leave check
sacOO7 Aug 14, 2023
d0a618c
Merge branch 'feature/integration-2.0' into fix/presencemap
sacOO7 Aug 15, 2023
a2e2d38
Added a new method to proto presence message to check for newer message
sacOO7 Aug 15, 2023
95b5ef8
Refactored code to handle synthesized and old message checks
sacOO7 Aug 15, 2023
ba8e1fc
Refactored realtime presence to make it easy to understand
sacOO7 Aug 15, 2023
d0694a4
Added spec annotation to presenceMessage newerThan method
sacOO7 Aug 16, 2023
2b84926
Added test for proto protocol message
sacOO7 Aug 16, 2023
c4122f4
commented out unused query variable, added tests for presence newness
sacOO7 Aug 16, 2023
b30a9f6
Added separate method to register error handler and published the same
sacOO7 Aug 16, 2023
2b151c1
Updated presence member action cases for enter, update and present
sacOO7 Aug 20, 2023
78d0a1a
Added method to realtime presence to add and remove presence member
sacOO7 Aug 20, 2023
f06f09e
Created shallow copy of presence messages to avoid presence action mu…
sacOO7 Aug 20, 2023
d40c5f6
For removed member, return false if existing member is absent
sacOO7 Aug 20, 2023
3cec497
Annotated realtime presence code with spec ids
sacOO7 Aug 20, 2023
78abf8f
Implemented spec RTP19 for leaving residual presence members
sacOO7 Aug 21, 2023
9bd5373
Added code to emit enter event from internal members
sacOO7 Aug 21, 2023
4412548
Refactored enter client presence for internal members, handled error …
sacOO7 Aug 22, 2023
16afe49
Updated err message for internal member presence enter error
sacOO7 Aug 22, 2023
8495354
Added spec implementation for realtime presence RTP5a/RTP5f
sacOO7 Aug 26, 2023
e94d526
Updated queuedMessages fail method with onlyPresenceMsg filter
sacOO7 Aug 26, 2023
993a7ff
Added spec implementation to handle presence msg queue on channel sta…
sacOO7 Aug 26, 2023
4e4a238
Checking channel states before processing presence messages
sacOO7 Aug 27, 2023
288ffb6
Updated send method for realtimepresence
sacOO7 Aug 28, 2023
a8c6290
refactored presence message send method
sacOO7 Aug 28, 2023
62a1513
Added a separate method for enqueing presence message
sacOO7 Aug 28, 2023
55bf6a4
Fixed enqueue messages in realtime_conn when queuing is not enabled
sacOO7 Aug 29, 2023
c41ddeb
Added a separate message queue to realtime presence
sacOO7 Aug 29, 2023
eeb86dc
Added warn log when transport level conn send error occurs
sacOO7 Aug 29, 2023
73f2823
Added realtime presence method to sendLocalPresenceQueueMessages
sacOO7 Aug 29, 2023
cefbf8d
Removed unnecessary implementation for queueFlush, sending all messag…
sacOO7 Aug 30, 2023
4cced5f
Refactored queue flush and fail method in state
sacOO7 Aug 30, 2023
e3c78e6
Removed unnecessary todo from realtime presence
sacOO7 Aug 30, 2023
9ff4f48
Fixed queue flush issues with realtime connection
sacOO7 Aug 30, 2023
6193f1b
Refactored realtime presence, annotated with right spec
sacOO7 Aug 30, 2023
25eebe5
Removed unnecessary inner message timestamp setting, already covered …
sacOO7 Aug 31, 2023
1fafafc
Refactored spec annotations for realtime presence
sacOO7 Aug 31, 2023
65faee6
Added missing spec annotations to presence spec
sacOO7 Aug 31, 2023
176bc4c
Refactored implementation for actionSync and actionPresence
sacOO7 Sep 2, 2023
83a54ca
Removed duplicate queue flush for channel state attached
sacOO7 Sep 2, 2023
4edf857
Added separate implementation for emitting an error update
sacOO7 Sep 3, 2023
542ff69
Extracted entering members from internal presence map into separate m…
sacOO7 Sep 3, 2023
22878b0
Refactored todo for processing in flight sync messages
sacOO7 Sep 3, 2023
f184caa
Updated implementation for syncSerial
sacOO7 Sep 6, 2023
7260161
Fixed spec annotations
sacOO7 Sep 6, 2023
7ee21ed
Removed unnecessary id param for internal map presence enter
sacOO7 Sep 6, 2023
74d2138
Updated realtime presence to flush queue on attached and enter members
sacOO7 Sep 12, 2023
3059000
Fixed failing tests for enter client presence
sacOO7 Oct 19, 2023
1544ee4
Updated realtime presence test for checking 250 members
sacOO7 Oct 23, 2023
ceed477
Updated test for sending message
sacOO7 Oct 24, 2023
6057ceb
Moved enterClient to realtime presence test, deleted old test file
sacOO7 Oct 24, 2023
1e92490
Refactored mock channel state for presence, removed unnecessary mock_…
sacOO7 Oct 25, 2023
233ca21
Merge branch 'feature/integration-2.0' into fix/presencemap
sacOO7 Oct 31, 2023
e10d895
annotated code for connection resume and recover impl
sacOO7 Oct 31, 2023
783d8a9
removed conn state closed from connection errors
sacOO7 Nov 1, 2023
cfa4e23
Annotated error reason for realtime conn
sacOO7 Nov 1, 2023
1c6276b
Removed unnecessary connection close test
sacOO7 Nov 2, 2023
e6db4a8
Added explicit error for connection unavailable
sacOO7 Nov 2, 2023
fa023db
Fixed proto protocol message test with msgSerial and connection Serial
sacOO7 Nov 3, 2023
6b4d944
Switched to new branch - msgFixtures for ably-common
sacOO7 Nov 3, 2023
22050e1
Updated common module in sync with main
sacOO7 Nov 3, 2023
6a08ee1
Formatted error.go file with missing errors
sacOO7 Nov 3, 2023
1e67fd5
Reformatted errors.go file using gofmt
sacOO7 Nov 3, 2023
4408c2f
refactored realtime conn, annotated with spec ids
sacOO7 Nov 4, 2023
c64a74f
Annotated test for RTN15c6
sacOO7 Nov 4, 2023
f045638
Updated integration test for RTNc3, replaced with RTNc7
sacOO7 Nov 5, 2023
e3c4336
Refactored connection resume tests related to RTN15c6 and RTN15c7
sacOO7 Nov 5, 2023
aa5633e
Added test to make sure channel goes into failed state
sacOO7 Nov 5, 2023
d1771c1
Fixed failing test for channel attach
sacOO7 Nov 5, 2023
515a654
refactored realtime channel spec integration for cancelled context
sacOO7 Nov 5, 2023
da19b34
Added missing cancelledCtx to test file
sacOO7 Nov 5, 2023
4137ff5
refactored state file, removed unnecessary msgCh struct
sacOO7 Nov 5, 2023
8d6e646
Revert "Added missing cancelledCtx to test file"
sacOO7 Nov 5, 2023
4875df9
Revert "refactored realtime channel spec integration for cancelled co…
sacOO7 Nov 5, 2023
3f390b7
Refactored cancelledContext for channel integration tests
sacOO7 Nov 5, 2023
0550ecc
Fixed failing test for realtime chan spec integration
sacOO7 Nov 5, 2023
0d8f475
Added lock safety for setting and retriving the channel serial
sacOO7 Nov 6, 2023
5bf488c
Added mtx lock for clientOptions recover
sacOO7 Nov 6, 2023
eb696f5
reverted mtx change for recover clientOptions, added explicit field t…
sacOO7 Nov 6, 2023
9751892
refactored code to access channelSerial based on locks
sacOO7 Nov 6, 2023
819a05c
Fixed realtime conn spec integration failing test
sacOO7 Nov 6, 2023
3a65ac6
Simplified assertions for realtime conn integration test
sacOO7 Nov 7, 2023
4c5c4cc
Refactored realtime channel code, added locks at right places
sacOO7 Nov 7, 2023
e2cd82e
setting recover only if recoveryContext connectionKey is set
sacOO7 Nov 7, 2023
55fbfc7
Added test assertions for RTN16
sacOO7 Nov 7, 2023
e383770
Improved retry mechanism while creating a testing sandbox app
sacOO7 Nov 7, 2023
8fbb62f
Updated test for connection recovery
sacOO7 Nov 7, 2023
822f4b9
Fixed intermittently failing tests
sacOO7 Nov 7, 2023
3d764eb
Fixed flaky test behavior for RTN16
sacOO7 Nov 7, 2023
d2910bd
Moved unit tests for presence to proto presence message test
sacOO7 Nov 8, 2023
3857e18
Fixed test failure to data race condition
sacOO7 Nov 8, 2023
70c7c71
Added one more check for connection retry for connection reset
sacOO7 Nov 8, 2023
be4a5a3
Updated connection recovery, added recover check for url query
sacOO7 Nov 8, 2023
e23b82a
Added dummy tests for RTP2 and RTP18
sacOO7 Nov 8, 2023
5e28c99
fixed data race condition for realtime presence
sacOO7 Nov 9, 2023
1e58236
Added dummy test for RTP2
sacOO7 Nov 9, 2023
5d5465f
Added basic test for RTP2
sacOO7 Nov 9, 2023
6473281
Added tests for RTP2 and RTP18
sacOO7 Nov 9, 2023
9dfe0ae
Added tests for RTP17
sacOO7 Nov 9, 2023
2e07f4c
Added test when message is synthesized, check newness by timestamp
sacOO7 Nov 9, 2023
1f6b109
Added test to check newness when msg not synthesized
sacOO7 Nov 9, 2023
7a9d7d4
Updated test to check newness by serial if not synthesized
sacOO7 Nov 12, 2023
b99a651
Fixed shallowcopy issue, added test for the same
sacOO7 Nov 12, 2023
30f7f10
Added a test for presence LEAVE action
sacOO7 Nov 12, 2023
8919bed
Added a test to check for newness during sync
sacOO7 Nov 12, 2023
2c088ac
Added tests for server initiated sync
sacOO7 Nov 12, 2023
1d587d0
Added has presence flag test for RTP1
sacOO7 Nov 12, 2023
f4e5c8a
Added test for internal presence map
sacOO7 Nov 13, 2023
7d36865
Updated failing tests due to locking mechanism
sacOO7 Nov 14, 2023
ace96a0
Fixed race condition while entering presence members from internal map
sacOO7 Nov 20, 2023
6443077
Fixed enter member from internal presence map data race issue
sacOO7 Nov 21, 2023
f0e50d0
Added assertion to check for enter action sent for internal members
sacOO7 Nov 21, 2023
f7afecd
Fixed goroutine issue for entering internal presence members
sacOO7 Nov 21, 2023
ca2eed1
Added extra check to avoid invoking goroutine on empty internal members
sacOO7 Nov 21, 2023
8b4f92c
Fixed data race issue for a failing test
sacOO7 Nov 21, 2023
bf8e196
Removed unnecessary check for channel state change when conn. in fail…
sacOO7 Nov 21, 2023
5865513
Fixed failing test for assertion
sacOO7 Nov 22, 2023
d2cc0cf
Refactored code as per review comments
sacOO7 Dec 1, 2023
2780d03
Removed OnError method on the channel, removed related references
sacOO7 Dec 3, 2023
31b739a
Refactored export_test file for presence members to avoid data race c…
sacOO7 Dec 4, 2023
14252e8
Updated syncWait method to include syncDone channel
sacOO7 Dec 4, 2023
e4ef0ce
Refactored syncmutex comments for realtime_presence
sacOO7 Dec 5, 2023
1b7a72d
Fixed race condition while accessing state, updated code as per spec
sacOO7 Dec 5, 2023
14cd5a2
Updated attached impl as per RTL5k
sacOO7 Dec 7, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions ably/error_names.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ const (
ErrTimeoutError ErrorCode = 50003
ErrConnectionFailed ErrorCode = 80000
ErrConnectionSuspended ErrorCode = 80002
ErrConnectionClosed ErrorCode = 80017
ErrDisconnected ErrorCode = 80003
ErrProtocolError ErrorCode = 80013
ErrChannelOperationFailed ErrorCode = 90000
Expand Down
12 changes: 12 additions & 0 deletions ably/errors.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

61 changes: 61 additions & 0 deletions ably/export_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,12 @@ func (c *REST) GetCachedFallbackHost() string {
return c.successFallbackHost.get()
}

func (c *RealtimeChannel) GetChannelSerial() string {
c.mtx.Lock()
defer c.mtx.Unlock()
return c.properties.ChannelSerial
}

func (c *RealtimeChannel) GetAttachResume() bool {
c.mtx.Lock()
defer c.mtx.Unlock()
Expand All @@ -105,6 +111,12 @@ func (c *RealtimeChannel) SetAttachResume(value bool) {
c.attachResume = value
}

func (c *RealtimeChannel) SetState(chanState ChannelState) {
c.mtx.Lock()
defer c.mtx.Unlock()
c.state = chanState
}

func (opts *clientOptions) GetFallbackRetryTimeout() time.Duration {
return opts.fallbackRetryTimeout()
}
Expand Down Expand Up @@ -193,6 +205,55 @@ func (c *Connection) PendingItems() int {
return len(c.pending.queue)
}

// AckAll empties queue and acks all pending callbacks
func (c *Connection) AckAll() {
c.mtx.Lock()
cx := c.pending.Dismiss()
c.mtx.Unlock()
c.log().Infof("Ack all %d messages waiting for ACK/NACK", len(cx))
for _, v := range cx {
v.onAck(nil)
}
}

func (c *Connection) SetKey(key string) {
c.mtx.Lock()
defer c.mtx.Unlock()
c.key = key
}

func (c *RealtimePresence) Members() map[string]*PresenceMessage {
c.mtx.Lock()
defer c.mtx.Unlock()
presenceMembers := make(map[string]*PresenceMessage, len(c.members))
for k, pm := range c.members {
presenceMembers[k] = pm
}
return presenceMembers
}

func (c *RealtimePresence) InternalMembers() map[string]*PresenceMessage {
c.mtx.Lock()
defer c.mtx.Unlock()
internalMembers := make(map[string]*PresenceMessage, len(c.internalMembers))
for k, pm := range c.internalMembers {
internalMembers[k] = pm
}
return internalMembers
}

func (c *RealtimePresence) SyncInitial() bool {
c.mtx.Lock()
defer c.mtx.Unlock()
return c.syncState == syncInitial
}

func (c *RealtimePresence) SyncInProgress() bool {
c.mtx.Lock()
defer c.mtx.Unlock()
return c.syncState == syncInProgress
}

func (c *Connection) ConnectionStateTTL() time.Duration {
return c.connectionStateTTL()
}
Expand Down
32 changes: 0 additions & 32 deletions ably/mock_test.go

This file was deleted.

1 change: 0 additions & 1 deletion ably/options.go
Original file line number Diff line number Diff line change
Expand Up @@ -235,7 +235,6 @@ func (opts *authOptions) KeySecret() string {
// clientOptions passes additional client-specific properties to the [ably.NewREST] or to the [ably.NewRealtime].
// Properties set using [ably.clientOptions] are used instead of the [ably.defaultOptions] values.
type clientOptions struct {

// authOptions Embedded an [ably.authOptions] object (TO3j).
authOptions

Expand Down
44 changes: 44 additions & 0 deletions ably/proto_presence_message.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ package ably

import (
"fmt"
"strconv"
"strings"
)

// PresenceAction describes the possible actions members in the presence set can emit (TP2).
Expand Down Expand Up @@ -58,3 +60,45 @@ func (m PresenceMessage) String() string {
"update",
}[m.Action], m.ClientID, m.Data)
}

func (msg *PresenceMessage) isServerSynthesized() bool {
return !strings.HasPrefix(msg.ID, msg.ConnectionID)
}

func (msg *PresenceMessage) getMsgSerialAndIndex() (int64, int64, error) {
msgIds := strings.Split(msg.ID, ":")
if len(msgIds) != 3 {
return 0, 0, fmt.Errorf("parsing error, the presence message has invalid id %v", msg.ID)
}
msgSerial, err := strconv.ParseInt(msgIds[1], 10, 64)
if err != nil {
return 0, 0, fmt.Errorf("parsing error, the presence message has invalid msgSerial, for msgId %v", msg.ID)
}
msgIndex, err := strconv.ParseInt(msgIds[2], 10, 64)
if err != nil {
return 0, 0, fmt.Errorf("parsing error, the presence message has invalid msgIndex, for msgId %v", msg.ID)
}
return msgSerial, msgIndex, nil
}

// RTP2b, RTP2c
func (msg1 *PresenceMessage) IsNewerThan(msg2 *PresenceMessage) (bool, error) {
// RTP2b1
if msg1.isServerSynthesized() || msg2.isServerSynthesized() {
return msg1.Timestamp > msg2.Timestamp, nil
}

// RTP2b2
msg1Serial, msg1Index, err := msg1.getMsgSerialAndIndex()
if err != nil {
return false, err
}
msg2Serial, msg2Index, err := msg2.getMsgSerialAndIndex()
if err != nil {
return true, err
}
if msg1Serial == msg2Serial {
return msg1Index > msg2Index, nil
}
return msg1Serial > msg2Serial, nil
}
Loading
Loading