Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Experimental] Stream Store #620

Merged
merged 15 commits into from
Jul 29, 2024
Merged

[Experimental] Stream Store #620

merged 15 commits into from
Jul 29, 2024

Conversation

DavidBadura
Copy link
Member

@DavidBadura DavidBadura commented Jul 24, 2024

This pull request introduces an event store that is decoupled from the aggregate root. This means that the columns aggregate and aggregateId have been merged into stream. Playhead is also optional in this store.

fix: #535

@DavidBadura DavidBadura force-pushed the stream-store branch 2 times, most recently from a8a9b71 to 8759f9a Compare July 24, 2024 10:04
Copy link

github-actions bot commented Jul 24, 2024

Hello 👋

here is the most recent benchmark result:

SimpleSetupBench
================

+----------------------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+
|                                        | time (kde mode)                                     | memory                                     |
+----------------------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+
| subject                                | Tag: <current>     | Tag: base          | time-diff | Tag: <current>  | Tag: base  | memory-diff |
+----------------------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+
| benchLoad1Event ()                     | 980.500μs (±0.00%) | 1.008ms (±0.00%)   | -2.75%    | 34.266mb        | 34.274mb   | -0.02%      |
| benchLoad10000Events ()                | 51.390ms (±0.00%)  | 50.933ms (±0.00%)  | +0.90%    | 33.764mb        | 33.764mb   | 0.00%       |
| benchSave1Event ()                     | 981.400μs (±0.00%) | 1.035ms (±0.00%)   | -5.17%    | 33.764mb        | 33.764mb   | 0.00%       |
| benchSave10000Events ()                | 229.509ms (±0.00%) | 238.039ms (±0.00%) | -3.58%    | 33.764mb        | 33.764mb   | 0.00%       |
| benchSave10000Aggregates ()            | 7.845s (±0.00%)    | 7.996s (±0.00%)    | -1.89%    | 33.764mb        | 33.764mb   | 0.00%       |
| benchSave10000AggregatesTransaction () | 4.752s (±0.00%)    | 4.731s (±0.00%)    | +0.45%    | 33.764mb        | 33.764mb   | 0.00%       |
+----------------------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+

SplitStreamBench
================

+-------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+
|                         | time (kde mode)                                     | memory                                     |
+-------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+
| subject                 | Tag: <current>     | Tag: base          | time-diff | Tag: <current>  | Tag: base  | memory-diff |
+-------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+
| benchLoad10000Events () | 4.545ms (±0.00%)   | 4.882ms (±0.00%)   | -6.89%    | 37.071mb        | 37.071mb   | 0.00%       |
| benchSave10000Events () | 341.039ms (±0.00%) | 355.189ms (±0.00%) | -3.98%    | 37.073mb        | 37.074mb   | -0.00%      |
+-------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+

SnapshotsBench
==============

+----------------------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+
|                                        | time (kde mode)                                     | memory                                     |
+----------------------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+
| subject                                | Tag: <current>     | Tag: base          | time-diff | Tag: <current>  | Tag: base  | memory-diff |
+----------------------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+
| benchLoad10000EventsMissingSnapshot () | 50.276ms (±0.00%)  | 49.738ms (±0.00%)  | +1.08%    | 33.765mb        | 33.765mb   | 0.00%       |
| benchLoad10000Events ()                | 897.500μs (±0.00%) | 904.800μs (±0.00%) | -0.81%    | 33.765mb        | 33.765mb   | 0.00%       |
+----------------------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+

PersonalDataBench
=================

+----------------------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+
|                                        | time (kde mode)                                     | memory                                     |
+----------------------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+
| subject                                | Tag: <current>     | Tag: base          | time-diff | Tag: <current>  | Tag: base  | memory-diff |
+----------------------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+
| benchLoad1Event ()                     | 889.300μs (±0.00%) | 853.800μs (±0.00%) | +4.16%    | 34.894mb        | 34.894mb   | 0.00%       |
| benchLoad10000Events ()                | 87.142ms (±0.00%)  | 83.719ms (±0.00%)  | +4.09%    | 34.894mb        | 34.894mb   | 0.00%       |
| benchSave1Event ()                     | 1.454ms (±0.00%)   | 1.416ms (±0.00%)   | +2.67%    | 34.894mb        | 34.894mb   | 0.00%       |
| benchSave10000Events ()                | 257.259ms (±0.00%) | 246.043ms (±0.00%) | +4.56%    | 34.896mb        | 34.896mb   | 0.00%       |
| benchSave10000Aggregates ()            | 12.177s (±0.00%)   | 11.876s (±0.00%)   | +2.54%    | 34.894mb        | 34.894mb   | 0.00%       |
| benchSave10000AggregatesTransaction () | 8.902s (±0.00%)    | 8.894s (±0.00%)    | +0.09%    | 35.395mb        | 35.395mb   | 0.00%       |
+----------------------------------------+--------------------+--------------------+-----------+-----------------+------------+-------------+

SubscriptionEngineBench
=======================

+---------------------------+-----------------+-----------------+-----------+-----------------+------------+-------------+
|                           | time (kde mode)                               | memory                                     |
+---------------------------+-----------------+-----------------+-----------+-----------------+------------+-------------+
| subject                   | Tag: <current>  | Tag: base       | time-diff | Tag: <current>  | Tag: base  | memory-diff |
+---------------------------+-----------------+-----------------+-----------+-----------------+------------+-------------+
| benchHandle10000Events () | 3.096s (±0.00%) | 3.154s (±0.00%) | -1.85%    | 34.163mb        | 34.163mb   | 0.00%       |
+---------------------------+-----------------+-----------------+-----------+-----------------+------------+-------------+

SimpleSetupStreamStoreBench
===========================

+----------------------------------------+--------------------+-----------+-----------------+-------------+
|                                        | time (kde mode)                | memory                        |
+----------------------------------------+--------------------+-----------+-----------------+-------------+
| subject                                | Tag: <current>     | time-diff | Tag: <current>  | memory-diff |
+----------------------------------------+--------------------+-----------+-----------------+-------------+
| benchLoad1Event ()                     | 944.400μs (±0.00%) | +0.00%    | 34.743mb        | +0.00%      |
| benchLoad10000Events ()                | 48.507ms (±0.00%)  | +0.00%    | 34.744mb        | +0.00%      |
| benchSave1Event ()                     | 1.009ms (±0.00%)   | +0.00%    | 34.743mb        | +0.00%      |
| benchSave10000Events ()                | 227.436ms (±0.00%) | +0.00%    | 34.744mb        | +0.00%      |
| benchSave10000Aggregates ()            | 7.766s (±0.00%)    | +0.00%    | 34.744mb        | +0.00%      |
| benchSave10000AggregatesTransaction () | 4.785s (±0.00%)    | +0.00%    | 34.744mb        | +0.00%      |
+----------------------------------------+--------------------+-----------+-----------------+-------------+

This comment gets update everytime a new commit comes in!

@DavidBadura DavidBadura added the enhancement New feature or request label Jul 24, 2024
@DavidBadura DavidBadura added this to the 3.4.0 milestone Jul 24, 2024
@DavidBadura DavidBadura changed the title POC: stream store Stream Store Jul 24, 2024
@DavidBadura DavidBadura marked this pull request as ready for review July 26, 2024 08:23
Comment on lines 112 to 119
private function streamName(AggregateHeader|StreamHeader $header): string
{
if ($header instanceof AggregateHeader) {
return StreamNameTranslator::streamName($header->aggregateName, $header->aggregateId);
}

return $header->streamName;
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe this should go into the StreamNameTranslator itself? Since this is done in another places in a different style

Copy link
Member Author

@DavidBadura DavidBadura Jul 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then we have an dependency aggregate->message.

Currently, the aggregate knows nothing about messages. (Except for the AggregateHeader, which will be removed in the long term)

In 4.0, StreamNameTranslator can also be removed again; it is mainly needed as a BC layer.
I want to solve this with metadata.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean domain wise? I mean, the Translator could also be moved into Store, which is imho even a better place, no?

I know that this translator acts as a BC layer, if the remove the "old" store in favor of the new StreamStore. I'm fine keeping it as is right now, but if we encounter a third place, i would vote for getting this more centralized.

Copy link
Member Author

@DavidBadura DavidBadura Jul 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How the stream name is composed is the task of the Aggregate/Repository domain. The store should not know anything about aggregate. In the future, only the repository will create this string. So, this is not a BC layer for the store, but a BC layer for the repository/aggregate which store it can work with.

I don't know if the Aggregate namespace is the right place. Maybe repository? But in my opinion, definitely not store.

docs/pages/store.md Show resolved Hide resolved
phpbench.json Show resolved Hide resolved
@DanielBadura
Copy link
Member

One last thing: Maybe we should add somewhere a note what to expect from a experimental feature? Like no BC compliance etc?

@DavidBadura
Copy link
Member Author

DavidBadura commented Jul 29, 2024

We should generally communicate how we handle our versions. Which versions are still supported, etc.

I opened an issue: #622

@DavidBadura DavidBadura merged commit c36ff32 into 3.4.x Jul 29, 2024
39 checks passed
@DavidBadura DavidBadura deleted the stream-store branch July 29, 2024 09:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Open event store for non-aggregate events
2 participants