Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature]It is recommended that ptubes support the separation of storage and computation #6

Open
myksl opened this issue May 20, 2022 · 0 comments
Labels
feature new features

Comments

@myksl
Copy link
Collaborator

myksl commented May 20, 2022

Describe the feature

I recently encountered a problem. Our business has some tasks with relatively large traffic, which will write a large amount of data. At the same time, because of business characteristics, we expect the data to be stored for a long time, which will cause our disk usage to be relatively high. In extreme cases will cause disk exhaustion
So, I hope to have a new ability

  1. Added a new StorageMode inside Reader, which supports S3 remote storage + Memory mode, and supports tiered storage
  2. Support Binlog persistent storage
  3. The real-time consumption stream maintains the same speed as before
  4. The local disk will be used as the real-time data layer, S3 will be used as the persistent storage layer, and the non-latest data of the local disk will be asynchronously written to S3 through certain rules
@myksl myksl added the feature new features label May 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature new features
Projects
None yet
Development

No branches or pull requests

1 participant