refactor: block serialisation and addition of tests #47
Conversation
WalkthroughThe pull request introduces several modifications across multiple files, primarily focusing on the structures and encoding processes related to blockchain data. Key changes include updates to the Changes
Possibly related PRs
Suggested reviewers
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 7
🧹 Outside diff range and nitpick comments (6)
dash/src/blockdata/transaction/special_transaction/coinbase.rs (1)
Line range hint
126-141: Consider adding edge cases to the test suite.While the current tests verify basic functionality and the size changes, consider adding test cases for:
- Maximum value for
best_cl_heightto verify compact size encoding handles large numbers correctly- Different
best_cl_heightvalues that would result in different compact size lengthsExample test cases to add:
// Test maximum height let payload = CoinbasePayload { height: 1000, version: 3, // ... other fields same as current test ... best_cl_height: Some(0xFFFFFFFF), // Max u32 value // ... remaining fields ... }; // Test height requiring different compact size lengths let test_heights = vec![0xFC, 0xFD, 0xFFFF, 0xFFFFFF]; for height in test_heights { let payload = CoinbasePayload { // ... same fields ... best_cl_height: Some(height), // ... remaining fields ... }; // ... size assertions ... }dash/src/blockdata/transaction/special_transaction/quorum_commitment.rs (2)
74-74: Simplify length calculation forsignersYou can replace
self.signers.iter().len()withself.signers.len()for clarity and potential performance benefits, aslen()is more direct and avoids creating an iterator.Apply this diff:
len += write_compact_size(w, self.signers.len() as u32)?; -len += write_fixed_bitset(w, self.signers.as_slice(), self.signers.iter().len())?; +len += write_fixed_bitset(w, self.signers.as_slice(), self.signers.len())?;
76-76: Simplify length calculation forvalid_membersSimilarly, replace
self.valid_members.iter().len()withself.valid_members.len()to make the code more concise.Apply this diff:
len += write_compact_size(w, self.valid_members.len() as u32)?; -len += write_fixed_bitset(w, self.valid_members.as_slice(), self.valid_members.iter().len())?; +len += write_fixed_bitset(w, self.valid_members.as_slice(), self.valid_members.len())?;dash/src/consensus/encode.rs (3)
1417-1460: Expand Tests to Cover Non-Minimal Encodings intest_compact_size_round_tripWhile the
test_compact_size_round_tripfunction tests valid round-trip serialization and deserialization, it doesn't test handling of non-minimal encodings.Consider adding test cases to verify that
read_compact_sizecorrectly rejects non-minimal encodings:#[test] fn test_compact_size_non_minimal_encodings() { // Non-minimal encoding: value 252 encoded with 3 bytes instead of 1 let data = vec![0xFD, 0xFC, 0x00]; let mut cursor = Cursor::new(&data); assert!(read_compact_size(&mut cursor).is_err()); // Non-minimal encoding: value 65535 encoded with 5 bytes instead of 3 let data = vec![0xFE, 0xFF, 0xFF, 0x00, 0x00]; let mut cursor = Cursor::new(&data); assert!(read_compact_size(&mut cursor).is_err()); }
35-35: Remove Unused ImportThe import statement
use std::io::Write;appears redundant because the traitWriteis already imported throughuse crate::io::{self, Cursor, Read};. This may lead to confusion or warnings about unused imports.Consider removing the redundant import:
-use std::io::Write;
1417-1515: Add Edge Case Tests fortest_fixed_bitset_round_tripThe
test_fixed_bitset_round_tripfunction covers several cases but could include more edge cases, such as whensizeis not a multiple of 8, or whenbitscontains alltrueor allfalsevalues.Consider adding the following test cases:
// Case where size is not a multiple of 8 and bits are all true let bits = vec![true; 13]; let size = 13; ... // Case where bits are all false let bits = vec![false; 20]; let size = 20; ...
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
📒 Files selected for processing (4)
dash/src/blockdata/block.rs(2 hunks)dash/src/blockdata/transaction/special_transaction/coinbase.rs(5 hunks)dash/src/blockdata/transaction/special_transaction/quorum_commitment.rs(6 hunks)dash/src/consensus/encode.rs(3 hunks)
🔇 Additional comments (10)
dash/src/blockdata/transaction/special_transaction/coinbase.rs (4)
26-26: LGTM: Import addition aligns with serialization refactoring.
The addition of compact size utilities from the consensus module is appropriate for the serialization changes.
47-48: LGTM: Size calculation improved with compact size encoding.
The size calculation now correctly accounts for variable-length encoding of best_cl_height using compact_size_len, making the serialization more space-efficient.
Also applies to: 52-56
71-71: LGTM: Encoding logic updated consistently.
The encoding now uses write_compact_size for best_cl_height, maintaining consistency with the size calculation changes while preserving proper error handling.
99-99: LGTM: Decoding logic updated symmetrically.
The decoding now uses read_compact_size for best_cl_height, maintaining symmetry with the encoding changes.
dash/src/blockdata/block.rs (2)
483-483: LGTM!
The updated hex value for work_bytes is correctly formatted and matches the expected size.
496-498: Please verify the test values.
The updated test values raise some concerns:
- The time value (1730283725) corresponds to a date in 2024, which seems unusually far in the future.
- The difficulty values (0 and 0.0037...) are unusually low for a blockchain block.
Could you please verify these values against the actual block data to ensure they are correct?
Also applies to: 504-505
✅ Verification successful
Let me gather more context about this test case to understand if these values are from actual block data.
Let me check the test data source to understand if these values are from actual test vectors.
Let me check the actual test data being used in this specific test case.
Based on the gathered context, I can now provide a final response:
The test values appear to be from actual block data and are correct
The test is validating block deserialization using real block data, where:
- The timestamp (1730283725) and other values are part of a complete test vector that includes matching prevhash, merkle root, and work values
- The low difficulty is consistent with the provided work value (0x...f7b6f1) in the test
- The test is properly structured to verify block header field serialization/deserialization
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Search for similar time and difficulty values in other test files
# to verify if these values are within expected ranges.
# Test: Search for time values in test files
rg -A 2 'time.*17[0-9]{8}' --type rust
# Test: Search for low difficulty values in test files
rg -A 2 'difficulty.*0\.00[0-9]+' --type rust
Length of output: 588
Script:
#!/bin/bash
# Look for the test function containing these assertions
rg -B 10 "assert_eq.*1730283725" --type rust
# Search for any block data or test vectors being used
rg -B 5 "real_decode.*header" --type rust
Length of output: 6978
Script:
#!/bin/bash
# Look for the test data/vectors being used in this test
rg -B 15 "some_block.*=" --type rust
# Also search for any constants or test data definitions
rg -B 5 "const.*BLOCK" --type rust
Length of output: 9360
Script:
#!/bin/bash
# Find the test function and its test data
rg -B 20 "1730283725" --type rust
Length of output: 3061
dash/src/blockdata/transaction/special_transaction/quorum_commitment.rs (3)
26-26: Imports for serialization utilities are appropriate
The added imports from consensus::encode are necessary for handling the encoding and decoding of compact sizes and fixed bitsets.
173-174: Verify lengths of signers and valid_members in test case
In this test case, signers has a length of 5, while valid_members has a length of 4. If these vectors are meant to be of the same length to represent quorum members consistently, this discrepancy might cause issues during serialization and deserialization.
195-196: Verify lengths of signers and valid_members in test case
Here, signers has a length of 7, and valid_members has a length of 6. Ensure that differing lengths are acceptable in the context of quorum commitments, or adjust the test cases to reflect intended usage.
dash/src/consensus/encode.rs (1)
908-931: Handle Unexpected EOF Gracefully in read_compact_size
If the reader does not contain enough bytes to fulfill the read requests, the function will return an UnexpectedEof error. Ensure that this behavior is acceptable or consider providing a more descriptive error message.
Run the following script to check how read_compact_size handles incomplete data:
Ensure that the function returns an appropriate error.
| pub signers: Vec<bool>, | ||
| pub valid_members: Vec<bool>, |
There was a problem hiding this comment.
Potential issues with Vec<bool> usage
Using Vec<bool> can lead to unexpected behaviors due to its internal bit-packing optimization. Consider using the bitvec crate's BitVec type for more robust and explicit bitset handling, or ensure that all operations account for the specifics of Vec<bool>.
dash/src/blockdata/transaction/special_transaction/quorum_commitment.rs
Outdated
Show resolved
Hide resolved
dash/src/consensus/encode.rs
Outdated
| pub fn write_fixed_bitset<W: Write + ?Sized>(w: &mut W, bits: &[bool], size: usize) -> io::Result<usize> { | ||
| // Calculate the number of bytes needed to represent 'size' bits | ||
| let num_bytes = (size + 7) / 8; | ||
| let mut bytes = vec![0u8; num_bytes]; | ||
|
|
||
| // Determine the minimum size to handle cases where bits.len() < size | ||
| let ms = std::cmp::min(size, bits.len()); | ||
|
|
||
| // Pack the bits into the byte buffer | ||
| for p in 0..ms { | ||
| if bits[p] { | ||
| bytes[p / 8] |= 1 << (p % 8); | ||
| } | ||
| } | ||
|
|
||
| // Write the bytes to the writer | ||
| w.write_all(&bytes)?; | ||
|
|
||
| // Return the number of bytes written | ||
| Ok(bytes.len()) | ||
| } |
There was a problem hiding this comment.
Validate size Parameter in write_fixed_bitset
Similarly, the write_fixed_bitset function uses the size parameter to determine buffer sizes without validation. If size is unreasonably large, it could cause memory issues.
Please add a validation check for size to ensure it's within acceptable bounds:
pub fn write_fixed_bitset<W: Write + ?Sized>(w: &mut W, bits: &[bool], size: usize) -> io::Result<usize> {
+ // Define a reasonable maximum size to prevent excessive memory allocation
+ const MAX_BITSET_SIZE: usize = 1_000_000; // Adjust this value as appropriate
+ if size > MAX_BITSET_SIZE {
+ return Err(io::Error::new(
+ io::ErrorKind::InvalidInput,
+ "Bitset size exceeds maximum allowed value",
+ ));
+ }
// Calculate the number of bytes needed to represent 'size' bits
let num_bytes = (size + 7) / 8;📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| pub fn write_fixed_bitset<W: Write + ?Sized>(w: &mut W, bits: &[bool], size: usize) -> io::Result<usize> { | |
| // Calculate the number of bytes needed to represent 'size' bits | |
| let num_bytes = (size + 7) / 8; | |
| let mut bytes = vec![0u8; num_bytes]; | |
| // Determine the minimum size to handle cases where bits.len() < size | |
| let ms = std::cmp::min(size, bits.len()); | |
| // Pack the bits into the byte buffer | |
| for p in 0..ms { | |
| if bits[p] { | |
| bytes[p / 8] |= 1 << (p % 8); | |
| } | |
| } | |
| // Write the bytes to the writer | |
| w.write_all(&bytes)?; | |
| // Return the number of bytes written | |
| Ok(bytes.len()) | |
| } | |
| pub fn write_fixed_bitset<W: Write + ?Sized>(w: &mut W, bits: &[bool], size: usize) -> io::Result<usize> { | |
| // Define a reasonable maximum size to prevent excessive memory allocation | |
| const MAX_BITSET_SIZE: usize = 1_000_000; // Adjust this value as appropriate | |
| if size > MAX_BITSET_SIZE { | |
| return Err(io::Error::new( | |
| io::ErrorKind::InvalidInput, | |
| "Bitset size exceeds maximum allowed value", | |
| )); | |
| } | |
| // Calculate the number of bytes needed to represent 'size' bits | |
| let num_bytes = (size + 7) / 8; | |
| let mut bytes = vec![0u8; num_bytes]; | |
| // Determine the minimum size to handle cases where bits.len() < size | |
| let ms = std::cmp::min(size, bits.len()); | |
| // Pack the bits into the byte buffer | |
| for p in 0..ms { | |
| if bits[p] { | |
| bytes[p / 8] |= 1 << (p % 8); | |
| } | |
| } | |
| // Write the bytes to the writer | |
| w.write_all(&bytes)?; | |
| // Return the number of bytes written | |
| Ok(bytes.len()) | |
| } |
Ensure Consistent bits Length Handling in write_fixed_bitset
In write_fixed_bitset, if bits.len() is less than size, the remaining bits are considered false. However, there's no indication to the caller that additional false bits were appended. This could lead to unexpected behavior.
Consider enforcing that bits.len() must be at least size or documenting this behavior clearly. Alternatively, you could return an error if bits.len() < size.
pub fn write_fixed_bitset<W: Write + ?Sized>(w: &mut W, bits: &[bool], size: usize) -> io::Result<usize> {
+ if bits.len() < size {
+ return Err(io::Error::new(
+ io::ErrorKind::InvalidInput,
+ "Bits length is less than the specified size",
+ ));
+ }Committable suggestion skipped: line range outside the PR's diff.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Outside diff range and nitpick comments (5)
dash/src/blockdata/transaction/special_transaction/quorum_commitment.rs (2)
73-76: Optimize length calculations in encoding.The
.iter().len()calls are redundant asVecalready implements thelen()method directly.Apply this diff to optimize the code:
-len += write_fixed_bitset(w, self.signers.as_slice(), self.signers.iter().len())?; +len += write_fixed_bitset(w, self.signers.as_slice(), self.signers.len())?; -len += write_fixed_bitset(w, self.valid_members.as_slice(), self.valid_members.iter().len())?; +len += write_fixed_bitset(w, self.valid_members.as_slice(), self.valid_members.len())?;
163-206: Enhance test coverage.While the size calculation tests are good, consider adding:
- Encoding/decoding roundtrip tests to verify data integrity
- Error case tests (e.g., invalid sizes, malformed input)
Would you like me to provide example test cases?
dash/src/consensus/encode.rs (3)
908-945: Consider using the custom Error type for consistency.The function returns
io::Result<u32>while other similar functions in this file use the customErrortype. Consider changing the return type toResult<u32, Error>for consistency.-pub fn read_compact_size<R: Read + ?Sized>(r: &mut R) -> io::Result<u32> { +pub fn read_compact_size<R: Read + ?Sized>(r: &mut R) -> Result<u32, Error> { let mut marker = [0u8; 1]; - r.read_exact(&mut marker)?; + r.read_exact(&mut marker).map_err(Error::Io)?; match marker[0] { 0xFD => { let mut buf = [0u8; 2]; - r.read_exact(&mut buf)?; + r.read_exact(&mut buf).map_err(Error::Io)?; let value = u16::from_le_bytes(buf) as u32; if value < 0xFD { - return Err(io::Error::new( - io::ErrorKind::InvalidData, - "Non-minimal compact size encoding", - )); + return Err(Error::NonMinimalVarInt); } Ok(value) }
966-978: Style and conciseness improvements suggested.The function has inconsistent spacing in else clauses and could be more concise.
pub fn compact_size_len(value: u32) -> usize { - let mut size: usize = 0; - if value < 253 { - size += 1; - } - else if value < 65536 { - size += 3; - } - else { - size += 5; - } - size + if value < 253 { + 1 + } else if value < 65536 { + 3 + } else { + 5 + } }
1007-1035: Add documentation about bits.len() < size behavior.The function correctly handles cases where
bits.len()is less thansize, but this behavior should be documented.Add a doc comment explaining that remaining bits are implicitly false:
+/// Writes a fixed-size bitset to a writer. +/// +/// # Arguments +/// * `w` - The writer to write to +/// * `bits` - The bits to write +/// * `size` - The size of the bitset +/// +/// If `bits.len()` is less than `size`, the remaining bits are treated as false. +/// +/// # Returns +/// The number of bytes written pub fn write_fixed_bitset<W: Write + ?Sized>(w: &mut W, bits: &[bool], size: usize) -> io::Result<usize> {
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
📒 Files selected for processing (3)
dash/src/blockdata/block.rs(2 hunks)dash/src/blockdata/transaction/special_transaction/quorum_commitment.rs(6 hunks)dash/src/consensus/encode.rs(3 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- dash/src/blockdata/block.rs
🔇 Additional comments (7)
dash/src/blockdata/transaction/special_transaction/quorum_commitment.rs (3)
26-26: LGTM: Import changes are appropriate.
The new imports for compact size and fixed bitset functions align with the refactoring changes.
39-40: Consider using bitvec crate instead of Vec<bool>.
51-54: LGTM: Size calculation is accurate.
The size calculation correctly accounts for both the compact size and fixed bitset lengths of the boolean vectors.
dash/src/consensus/encode.rs (4)
947-964: LGTM!
The implementation correctly handles different size ranges with proper byte ordering and clear comments.
980-1005: LGTM!
The implementation correctly handles size validation and bit manipulation, with proper memory allocation limits.
1037-1043: LGTM!
The implementation correctly calculates the required bytes for the bitset with clear variable naming.
1447-1545: LGTM! Comprehensive test coverage.
The test suite provides thorough coverage of both normal cases and edge cases, with well-organized test cases and proper round-trip testing.
compact_size_len,write_compact_size,read_compact_size,fixed_bitset_len,write_fixed_bitsetandread_fixed_bitsetSummary by CodeRabbit
New Features
CoinbasePayloadandQuorumFinalizationCommitmentto improve data representation and processing.Bug Fixes
Documentation