feat: Matching algo performance + Exclude duplicate matchings #1142
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 3 commits
September 14, 2024 23:04
Run the matching algorithm every n days while simulating match requests of real users.
Jonasdoubleyou
temporarily deployed
to
backend-feat-matching-a-i58zxe
Jonasdoubleyou
had a problem deploying
to
backend-feat-matching-a-i58zxe
Jonasdoubleyou
temporarily deployed
to
backend-feat-matching-a-i58zxe
Jonasdoubleyou
had a problem deploying
to
backend-feat-matching-a-i58zxe
Jonasdoubleyou
changed the title
Jonasdoubleyou
temporarily deployed
to
backend-feat-matching-a-i58zxe
Jonasdoubleyou
temporarily deployed
to
backend-feat-matching-a-i58zxe
Jonasdoubleyou
temporarily deployed
to
backend-feat-matching-a-i58zxe
Jonasdoubleyou
temporarily deployed
to
backend-feat-matching-a-i58zxe
Jonasdoubleyou
temporarily deployed
to
backend-feat-matching-a-i58zxe
Jonasdoubleyou
temporarily deployed
to
backend-feat-matching-a-i58zxe
Jonasdoubleyou
temporarily deployed
to
backend-feat-matching-a-i58zxe
Jonasdoubleyou
temporarily deployed
to
backend-feat-matching-a-i58zxe
As can be seen from the diff, this only has a small impact on the matched subjects while improving the matching state rate from 10% to 30%.
Keep pupils a bit longer in the match pool to find a better match. This increases wait time from 4 days to 17 days on average, but increases both the number of subjects and states matched. Not sure if the trade-off is worth it, but now we have a mechanism for it that can be tuned.
Jonasdoubleyou
temporarily deployed
to
backend-feat-matching-a-i58zxe
Jonasdoubleyou
temporarily deployed
to
backend-feat-matching-a-i58zxe
Jonasdoubleyou
commented
Sep 15, 2024
| 'new', | ||
| 1, | ||
| { | ||
| matchCountSum: 1044, |
There was a problem hiding this comment.
As the new algorithm is able to create more matches with the same "quality" (about the same number of matching subjects and states) I think it is worth to switch.
| 'old', | ||
| 1000, | ||
| { | ||
| matchCountSum: 883, |
There was a problem hiding this comment.
It's interesting that the old algorithm prioritizes the number of matching states over the number of matching subjects and the number of matches created ...
| [ | ||
| 'new', | ||
| 1000, | ||
| { |
There was a problem hiding this comment.
Using such a fixed time series as a benchmark is not so optimal - it can easily lead to overfitting onto this time series. Some sort of shuffling of the input would be good, but on the other hand that might loose some time effects that are actually there (e.g. pupils / students signing up in bulk after marketing campaigns)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Splits the tests into small unit tests that are fast to run (
npm run unit) and perf tests that might take longer (npm run perf) -npm run testruns all of them in the barrier.Adds a performance test for the matching algorithm, which simulates a real world matching - pupils and students continously request matches, every N days the matching algorithm is run using the current pools, matched users are removed from the pool. At the end, reports how many matches are created, how many subjects and states match, and how long the pupils and students had to wait for the match.
Extends the Matching Algorithm to exclude duplicate matches (as the old implementation does). Currently implemented as a postfilter after the matching, dont think it happens often and is worth considering in the assignment itself.
https://github.com/corona-school/project-user/issues/1316