Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: secondary rocksdb storage #281

Draft
wants to merge 7 commits into
base: develop
Choose a base branch
from
Draft

Conversation

shumkov
Copy link
Member

@shumkov shumkov commented Feb 8, 2024

Issue being fixed or feature implemented

What was done?

How Has This Been Tested?

Breaking Changes

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have made corresponding changes to the documentation

For repository code-owners and collaborators only

  • I have assigned this pull request to a milestone

grovedb/src/lib.rs Show resolved Hide resolved
grovedb/tests/secondary_tests.rs Outdated Show resolved Hide resolved
storage/src/rocksdb_storage/storage.rs Show resolved Hide resolved

/// Replicate recent changes from primary database
/// Available only for a secondary storage
pub fn try_to_catch_up_from_primary(&self) -> Result<(), Error> {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've rarely seen any try_ naming except try_from and only because there is a non-try alternative (just from), the Result in the signature is explicit enough about possible failure

storage/src/rocksdb_storage/storage.rs Outdated Show resolved Hide resolved

self.commit_db_write_batch(db_batch, pending_costs, transaction)
.add_cost(cost)
fn create_checkpoint<P: AsRef<Path>>(&self, path: P) -> Result<(), Error> {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shall we even checkpoint on secondary?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technically you can do it. Do you think we shouldn't allow it?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know how you plan to split responsibilities between instances, and whether it works for secondary instances, it just caught my eye

pub struct PrefixedSecondaryRocksDbStorageContext<'db> {
pub(in crate::rocksdb_storage) storage: &'db NonTransactionalDb,
pub(in crate::rocksdb_storage) prefix: SubtreePrefix,
pub(in crate::rocksdb_storage) batch: Option<&'db StorageBatch>,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need to have it for secondary?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We reuse it in another context

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do you mean

}

/// Create a new prefixed context instance
pub fn new_secondary(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's context_tx file, we're dealing with transactions there and this one receives none as it should

so I suggest to have no such context at all because we have no such case for secondary

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants