Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add rbx_binary Miner's Haven benchmark and readme #473

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions rbx_binary/benches/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# `rbx_binary` Benchmark Suite

This directory contains a suite of benchmarks used to measure the performance of the `rbx_binary` serializer and deserializer.

### Adding a new benchmark

To add a new benchmark, first add the file you'd like to measure performance against to the `files` directory. Then, add a new benchmark function to `suite/main.rs`, like this:
```rust
pub fn my_benchmark(c: &mut Criterion) {
bench(
&mut c.benchmark_group("My Benchmark")
include_bytes!("../files/bench-file.rbxl"),
)
}
```
and also make sure to add your benchmark function to the `criterion_group!` macro invocation.

Benchmark groups provide a number of configuration options which are useful under different circumstances. See the [Criterion.rs benchmark configuration documentation](https://bheisler.github.io/criterion.rs/book/user_guide/advanced_configuration.html) for details.

### Running the benchmarks

To run all benchmarks, run the following command somewhere in the `rbx_binary` crate directory:
```bash
cargo bench
```

To run a specific benchmark, run the following command somewhere in the `rbx_binary` crate directory, subsituting `My Benchmark` with the name of the benchmark group:
```bash
cargo bench "My Benchmark"
```

To measure only serialization or deserialization, add `/Serialize` or `/Deserialize` to the end of the benchmark name, like this:
```bash
cargo bench "My Benchmark/Serialize"
```

Once the benchmark is complete, an HTML report will be generated at `rbx-dom/target/criterion/reports/index.html` that contains detailed statistics collected during the run. This file can be opened in a web browser.

For more information, see the [Criterion.rs documentation](https://bheisler.github.io/criterion.rs/book/).
Binary file added rbx_binary/benches/files/miners-haven.rbxl
Binary file not shown.
9 changes: 9 additions & 0 deletions rbx_binary/benches/suite/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -32,12 +32,21 @@ pub fn parts_1000(c: &mut Criterion) {
)
}

pub fn miners_haven(c: &mut Criterion) {
bench(
c.benchmark_group("Miner's Haven")
.sampling_mode(SamplingMode::Flat),
include_bytes!("../files/miners-haven.rbxl"),
)
}

criterion_group!(
bench_suite,
folders_100,
deep_folders_100,
modulescripts_100_lines_100,
parts_1000,
miners_haven,
);

criterion_main!(bench_suite);
7 changes: 6 additions & 1 deletion rbx_binary/benches/suite/util.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,11 @@ fn serialize_bench<T: Measurement>(group: &mut BenchmarkGroup<T>, buffer: &[u8])

rbx_binary::to_writer(&mut buffer, &tree, &[root_ref]).unwrap();
let buffer_len = buffer.len();
let batch_size = if buffer_len > 1024 {
BatchSize::LargeInput
} else {
BatchSize::SmallInput
};

group
.throughput(Throughput::Bytes(buffer_len as u64))
Expand All @@ -21,7 +26,7 @@ fn serialize_bench<T: Measurement>(group: &mut BenchmarkGroup<T>, buffer: &[u8])
|mut buffer: Vec<u8>| {
rbx_binary::to_writer(&mut buffer, &tree, &[root_ref]).unwrap();
},
BatchSize::SmallInput,
batch_size,
)
});
}
Expand Down