-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CV2-4719 initial idea saround caching layer in presto #92
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
code looks great to me. Did we answer Caio's question about what the actual requests look like from submitting a media item? i.e. do we see the three requests, do they all have a sha we can use? Are they far enough apart in time that we will have cached the result of the first query when the second arrives?
I've tested out the requests directly and confirmed that wherever possible we set a SHA in Check-API - I am not sure about the timing aspect of all of this, however - it could very likely be that we will not actually end up using the queue because requests are sent too quickly to have the second one hit the cache - I think we should solve that in a separate ticket however once we've actually put this out into production, as that's thematically a bit of a different question about how we can restructure the request architecture in Check-API. |
maybe could add some logging or a metric so we will know what % of requests use the cache? |
Good idea- I'll put that together now! |
Description
Adds a redis caching layer to presto to optionally bypass the fingerprinting routine for already-fingerprinted items. Also adds ability to flexibly define ttl and optionally bypass that.
Reference: CV2-4719
How has this been tested?
Not yet tested locally - will do integration testing once unit tests passing on all three services
Are there any external dependencies?
Adds redis - will need to make sure we can access it with @sonoransun
Have you considered secure coding practices when writing this code?
Nothing particularly interesting here