Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

IPA runs with 3M input fails with ZKP batcher assertion error #1308

Open
akoshelev opened this issue Sep 23, 2024 · 4 comments
Open

IPA runs with 3M input fails with ZKP batcher assertion error #1308

akoshelev opened this issue Sep 23, 2024 · 4 comments
Assignees
@andyleiserson

Comments

@akoshelev
Copy link
Collaborator

Link to the run: https://draft-mpc.vercel.app/query/view/adult-sum2024-09-23T1639

2024-09-23T17:36:39.945Z | Helper3:
assertion failed: self.batches.len() <= 1
2024-09-23T17:36:39.932Z | Helper1:
2024-09-23T17:36:39.929625Z ERROR oprf_ipa_query{sz=3000000}:attribute_cap_aggregate: ipa_core::error: ThreadId(31) "tokio-runtime-worker" panicked at ipa-core/src/protocol/context/batcher.rs:251:9:
2024-09-23T17:36:39.933Z | Helper2:
2024-09-23T17:36:39.929600Z ERROR oprf_ipa_query{sz=3000000}:attribute_cap_aggregate: ipa_core::error: ThreadId(31) "tokio-runtime-worker" panicked at ipa-core/src/protocol/context/batcher.rs:251:9:
2024-09-23T17:36:39.945Z | Helper3:
assertion failed: self.batches.len() <= 1
2024-09-23T17:36:39.932Z | Helper1:
2024-09-23T17:36:39.929625Z ERROR oprf_ipa_query{sz=3000000}:attribute_cap_aggregate: ipa_core::error: ThreadId(31) "tokio-runtime-worker" panicked at ipa-core/src/protocol/context/batcher.rs:251:9:
2024-09-23T17:36:39.945Z | Helper1:
assertion failed: self.batches.len() <= 1
@akoshelev
Copy link
Collaborator Author

@andyleiserson pointed out that we could set the batch size to an artificially large number (1B) to get past this error.

    let validator = sh_ctx.dzkp_validator(
        MaliciousProtocolSteps {
            protocol: &Step::Aggregate,
            validate: &Step::AggregateValidate,
        },
        // 1B batch size, suboptimal. But only to test that it works for 3M
        1 << 30,
    );

I did that and got past 3M, but the same error (maybe from a different callsite) occurs at 4M
https://draft-mpc.vercel.app/query/view/rust-hack2024-09-24T0615

@akoshelev
Copy link
Collaborator Author 

2024-09-24T06:55:58.312Z | Helper1:
2024-09-24T06:55:58.308982Z INFO oprf_ipa_query{sz=4000000}:attribute_cap_aggregate:breakdown_reveal_aggregation{total=3272801}: ipa_core::protocol::ipa_prf::aggregation::breakdown_reveal: close time.busy=85.2s time.idle=196s
2024-09-24T06:55:58.312Z | Helper1:
2024-09-24T06:55:58.308982Z INFO oprf_ipa_query{sz=4000000}:attribute_cap_aggregate:breakdown_reveal_aggregation{total=3272801}: ipa_core::protocol::ipa_prf::aggregation::breakdown_reveal: close time.busy=85.2s time.idle=196s
2024-09-24T06:55:58.312Z | Helper1:
2024-09-24T06:55:58.309082Z ERROR oprf_ipa_query{sz=4000000}:attribute_cap_aggregate: ipa_core::error: ThreadId(31) "tokio-runtime-worker" panicked at ipa-core/src/protocol/context/batcher.rs:251:9:
2024-09-24T06:55:58.329Z | Helper1:
assertion failed: self.batches.len() <= 1
2024-09-24T06:55:58.312Z | Helper1:
2024-09-24T06:55:58.309082Z ERROR oprf_ipa_query{sz=4000000}:attribute_cap_aggregate: ipa_core::error: ThreadId(31) "tokio-runtime-worker" panicked at ipa-core/src/protocol/context/batcher.rs:251:9:
2024-09-24T06:55:58.312Z | Helper1:
2024-09-24T06:55:58.309082Z ERROR oprf_ipa_query{sz=4000000}:attribute_cap_aggregate: ipa_core::error: ThreadId(31) "tokio-runtime-worker" panicked at ipa-core/src/protocol/context/batcher.rs:251:9:
2024-09-24T06:55:58.329Z | Helper1:
assertion failed: self.batches.len() <= 1
2024-09-24T06:55:58.329Z | Helper1:
stack trace:
2024-09-24T06:55:58.329Z | Helper1:
assertion failed: self.batches.len() <= 1
2024-09-24T06:55:58.329Z | Helper1:
assertion failed: self.batches.len() <= 1
2024-09-24T06:55:58.329Z | Helper1:
assertion failed: self.batches.len() <= 1
2024-09-24T06:55:58.329Z | Helper1:
stack trace:
2024-09-24T06:55:58.329Z | Helper1:
assertion failed: self.batches.len() <= 1
2024-09-24T06:55:58.329Z | Helper1:
stack trace:
2024-09-24T06:55:58.329Z | Helper1:
0: ipa_core::error::set_global_panic_hook::{{closure}}
2024-09-24T06:55:58.330Z | Helper1:
1: std::panicking::rust_panic_with_hook
2024-09-24T06:55:58.329Z | Helper1:
0: ipa_core::error::set_global_panic_hook::{{closure}}
2024-09-24T06:55:58.330Z | Helper1:
1: std::panicking::rust_panic_with_hook
2024-09-24T06:55:58.329Z | Helper1:
0: ipa_core::error::set_global_panic_hook::{{closure}}
2024-09-24T06:55:58.329Z | Helper1:
0: ipa_core::error::set_global_panic_hook::{{closure}}
2024-09-24T06:55:58.330Z | Helper1:
1: std::panicking::rust_panic_with_hook
2024-09-24T06:55:58.330Z | Helper1:
2: std::panicking::begin_panic_handler::{{closure}}
2024-09-24T06:55:58.330Z | Helper1:
1: std::panicking::rust_panic_with_hook
2024-09-24T06:55:58.330Z | Helper1:
1: std::panicking::rust_panic_with_hook
2024-09-24T06:55:58.330Z | Helper1:
2: std::panicking::begin_panic_handler::{{closure}}
2024-09-24T06:55:58.330Z | Helper1:
3: std::sys_common::backtrace::__rust_end_short_backtrace
2024-09-24T06:55:58.330Z | Helper1:
2: std::panicking::begin_panic_handler::{{closure}}
2024-09-24T06:55:58.330Z | Helper1:
2: std::panicking::begin_panic_handler::{{closure}}
2024-09-24T06:55:58.330Z | Helper1:
3: std::sys_common::backtrace::__rust_end_short_backtrace
2024-09-24T06:55:58.330Z | Helper1:
4: rust_begin_unwind
2024-09-24T06:55:58.330Z | Helper1:
3: std::sys_common::backtrace::__rust_end_short_backtrace
2024-09-24T06:55:58.330Z | Helper1:
3: std::sys_common::backtrace::__rust_end_short_backtrace
2024-09-24T06:55:58.330Z | Helper1:
4: rust_begin_unwind
2024-09-24T06:55:58.331Z | Helper1:
5: core::panicking::panic_fmt
2024-09-24T06:55:58.330Z | Helper1:
4: rust_begin_unwind
2024-09-24T06:55:58.330Z | Helper1:
4: rust_begin_unwind
2024-09-24T06:55:58.331Z | Helper1:
5: core::panicking::panic_fmt
2024-09-24T06:55:58.331Z | Helper1:
6: core::panicking::panic
2024-09-24T06:55:58.331Z | Helper1:
5: core::panicking::panic_fmt
2024-09-24T06:55:58.331Z | Helper1:
5: core::panicking::panic_fmt
2024-09-24T06:55:58.331Z | Helper1:
6: core::panicking::panic
2024-09-24T06:55:58.331Z | Helper1:
7: ipa_core::protocol::context::batcher::Batcher<B>::into_single_batch
2024-09-24T06:55:58.331Z | Helper1:
6: core::panicking::panic
2024-09-24T06:55:58.331Z | Helper1:
6: core::panicking::panic
2024-09-24T06:55:58.331Z | Helper1:
6: core::panicking::panic
2024-09-24T06:55:58.331Z | Helper1:
7: ipa_core::protocol::context::batcher::Batcher<B>::into_single_batch
2024-09-24T06:55:58.331Z | Helper1:
8: <ipa_core::protocol::context::dzkp_validator::MaliciousDZKPValidator as ipa_core::protocol::context::dzkp_validator::DZKPValidator>::validate::{{closure}}
2024-09-24T06:55:58.331Z | Helper1:
7: ipa_core::protocol::context::batcher::Batcher<B>::into_single_batch
2024-09-24T06:55:58.331Z | Helper1:
8: <ipa_core::protocol::context::dzkp_validator::MaliciousDZKPValidator as ipa_core::protocol::context::dzkp_validator::DZKPValidator>::validate::{{closure}}
2024-09-24T06:55:58.331Z | Helper1:
7: ipa_core::protocol::context::batcher::Batcher<B>::into_single_batch
2024-09-24T06:55:58.331Z | Helper1:
7: ipa_core::protocol::context::batcher::Batcher<B>::into_single_batch
2024-09-24T06:55:58.331Z | Helper1:
8: <ipa_core::protocol::context::dzkp_validator::MaliciousDZKPValidator as ipa_core::protocol::context::dzkp_validator::DZKPValidator>::validate::{{closure}}
2024-09-24T06:55:58.332Z | Helper1:
9: ipa_core::protocol::ipa_prf::prf_sharding::attribute_cap_aggregate::{{closure}}::{{closure}}
2024-09-24T06:55:58.331Z | Helper1:
8: <ipa_core::protocol::context::dzkp_validator::MaliciousDZKPValidator as ipa_core::protocol::context::dzkp_validator::DZKPValidator>::validate::{{closure}}
2024-09-24T06:55:58.331Z | Helper1:
8: <ipa_core::protocol::context::dzkp_validator::MaliciousDZKPValidator as ipa_core::protocol::context::dzkp_validator::DZKPValidator>::validate::{{closure}}
2024-09-24T06:55:58.331Z | Helper1:
8: <ipa_core::protocol::context::dzkp_validator::MaliciousDZKPValidator as ipa_core::protocol::context::dzkp_validator::DZKPValidator>::validate::{{closure}}
2024-09-24T06:55:58.332Z | Helper1:
9: ipa_core::protocol::ipa_prf::prf_sharding::attribute_cap_aggregate::{{closure}}::{{closure}}
2024-09-24T06:55:58.332Z | Helper1:
10: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll
2024-09-24T06:55:58.332Z | Helper1:
9: ipa_core::protocol::ipa_prf::prf_sharding::attribute_cap_aggregate::{{closure}}::{{closure}}
2024-09-24T06:55:58.332Z | Helper1:
9: ipa_core::protocol::ipa_prf::prf_sharding::attribute_cap_aggregate::{{closure}}::{{closure}}
2024-09-24T06:55:58.332Z | Helper1:
10: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll
2024-09-24T06:55:58.332Z | Helper1:
11: ipa_core::protocol::ipa_prf::oprf_ipa::{{closure}}
2024-09-24T06:55:58.332Z | Helper1:
10: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll
2024-09-24T06:55:58.332Z | Helper1:
11: ipa_core::protocol::ipa_prf::oprf_ipa::{{closure}}
2024-09-24T06:55:58.332Z | Helper1:
10: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll
2024-09-24T06:55:58.332Z | Helper1:
10: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll
2024-09-24T06:55:58.332Z | Helper1:
10: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll
2024-09-24T06:55:58.332Z | Helper1:
11: ipa_core::protocol::ipa_prf::oprf_ipa::{{closure}}
2024-09-24T06:55:58.332Z | Helper1:
12: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll
2024-09-24T06:55:58.332Z | Helper1:
11: ipa_core::protocol::ipa_prf::oprf_ipa::{{closure}}
2024-09-24T06:55:58.332Z | Helper1:
11: ipa_core::protocol::ipa_prf::oprf_ipa::{{closure}}
2024-09-24T06:55:58.332Z | Helper1:
12: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll
2024-09-24T06:55:58.333Z | Helper1:
13: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll
2024-09-24T06:55:58.332Z | Helper1:
12: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll
2024-09-24T06:55:58.332Z | Helper1:
12: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll
2024-09-24T06:55:58.333Z | Helper1:
13: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll
2024-09-24T06:55:58.333Z | Helper1:
14: <futures_util::future::future::flatten::Flatten<Fut,<Fut as core::future::future::Future>::Output> as core::future::future::Future>::poll
2024-09-24T06:55:58.333Z | Helper1:
13: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll
2024-09-24T06:55:58.333Z | Helper1:
13: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll
2024-09-24T06:55:58.333Z | Helper1:
13: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll
2024-09-24T06:55:58.333Z | Helper1:
14: <futures_util::future::future::flatten::Flatten<Fut,<Fut as core::future::future::Future>::Output> as core::future::future::Future>::poll
2024-09-24T06:55:58.333Z | Helper1:
15: <futures_util::future::future::Then<Fut1,Fut2,F> as core::future::future::Future>::poll
2024-09-24T06:55:58.333Z | Helper1:
14: <futures_util::future::future::flatten::Flatten<Fut,<Fut as core::future::future::Future>::Output> as core::future::future::Future>::poll
2024-09-24T06:55:58.333Z | Helper1:
15: <futures_util::future::future::Then<Fut1,Fut2,F> as core::future::future::Future>::poll
2024-09-24T06:55:58.333Z | Helper1:
14: <futures_util::future::future::flatten::Flatten<Fut,<Fut as core::future::future::Future>::Output> as core::future::future::Future>::poll
2024-09-24T06:55:58.333Z | Helper1:
15: <futures_util::future::future::Then<Fut1,Fut2,F> as core::future::future::Future>::poll
2024-09-24T06:55:58.333Z | Helper1:
16: tokio::runtime::park::CachedParkThread::block_on
2024-09-24T06:55:58.333Z | Helper1:
15: <futures_util::future::future::Then<Fut1,Fut2,F> as core::future::future::Future>::poll
2024-09-24T06:55:58.333Z | Helper1:
15: <futures_util::future::future::Then<Fut1,Fut2,F> as core::future::future::Future>::poll
2024-09-24T06:55:58.333Z | Helper1:
16: tokio::runtime::park::CachedParkThread::block_on
2024-09-24T06:55:58.334Z | Helper1:
17: tokio::runtime::context::runtime::enter_runtime
2024-09-24T06:55:58.333Z | Helper1:
16: tokio::runtime::park::CachedParkThread::block_on
2024-09-24T06:55:58.333Z | Helper1:
16: tokio::runtime::park::CachedParkThread::block_on
2024-09-24T06:55:58.333Z | Helper1:
16: tokio::runtime::park::CachedParkThread::block_on
2024-09-24T06:55:58.334Z | Helper1:
17: tokio::runtime::context::runtime::enter_runtime
2024-09-24T06:55:58.334Z | Helper1:
18: tokio::runtime::context::runtime_mt::exit_runtime
2024-09-24T06:55:58.334Z | Helper1:
17: tokio::runtime::context::runtime::enter_runtime
2024-09-24T06:55:58.334Z | Helper1:
17: tokio::runtime::context::runtime::enter_runtime
2024-09-24T06:55:58.334Z | Helper1:
18: tokio::runtime::context::runtime_mt::exit_runtime
2024-09-24T06:55:58.334Z | Helper1:
19: tokio::runtime::scheduler::multi_thread::worker::block_in_place
2024-09-24T06:55:58.334Z | Helper1:
18: tokio::runtime::context::runtime_mt::exit_runtime
2024-09-24T06:55:58.334Z | Helper1:
18: tokio::runtime::context::runtime_mt::exit_runtime
2024-09-24T06:55:58.334Z | Helper1:
19: tokio::runtime::scheduler::multi_thread::worker::block_in_place
2024-09-24T06:55:58.334Z | Helper1:
20: ipa_core::query::executor::do_query::{{closure}}
2024-09-24T06:55:58.334Z | Helper1:
19: tokio::runtime::scheduler::multi_thread::worker::block_in_place
2024-09-24T06:55:58.334Z | Helper1:
20: ipa_core::query::executor::do_query::{{closure}}
2024-09-24T06:55:58.334Z | Helper1:
19: tokio::runtime::scheduler::multi_thread::worker::block_in_place
2024-09-24T06:55:58.334Z | Helper1:
19: tokio::runtime::scheduler::multi_thread::worker::block_in_place
2024-09-24T06:55:58.334Z | Helper1:
20: ipa_core::query::executor::do_query::{{closure}}
2024-09-24T06:55:58.334Z | Helper1:
21: tokio::runtime::task::core::Core<T,S>::poll
2024-09-24T06:55:58.334Z | Helper1:
20: ipa_core::query::executor::do_query::{{closure}}
2024-09-24T06:55:58.334Z | Helper1:
20: ipa_core::query::executor::do_query::{{closure}}
2024-09-24T06:55:58.334Z | Helper1:
21: tokio::runtime::task::core::Core<T,S>::poll
2024-09-24T06:55:58.334Z | Helper1:
20: ipa_core::query::executor::do_query::{{closure}}
2024-09-24T06:55:58.334Z | Helper1:
21: tokio::runtime::task::core::Core<T,S>::poll
2024-09-24T06:55:58.335Z | Helper1:
22: tokio::runtime::task::harness::Harness<T,S>::poll
2024-09-24T06:55:58.335Z | Helper1:
23: tokio::runtime::scheduler::multi_thread::worker::Context::run_task
2024-09-24T06:55:58.335Z | Helper1:
22: tokio::runtime::task::harness::Harness<T,S>::poll
2024-09-24T06:55:58.335Z | Helper1:
22: tokio::runtime::task::harness::Harness<T,S>::poll
2024-09-24T06:55:58.335Z | Helper1:
23: tokio::runtime::scheduler::multi_thread::worker::Context::run_task
2024-09-24T06:55:58.335Z | Helper1:
24: tokio::runtime::scheduler::multi_thread::worker::Context::run
2024-09-24T06:55:58.335Z | Helper1:
23: tokio::runtime::scheduler::multi_thread::worker::Context::run_task
2024-09-24T06:55:58.335Z | Helper1:
23: tokio::runtime::scheduler::multi_thread::worker::Context::run_task
2024-09-24T06:55:58.335Z | Helper1:
24: tokio::runtime::scheduler::multi_thread::worker::Context::run
2024-09-24T06:55:58.335Z | Helper1:
25: tokio::runtime::context::runtime::enter_runtime
2024-09-24T06:55:58.335Z | Helper1:
24: tokio::runtime::scheduler::multi_thread::worker::Context::run
2024-09-24T06:55:58.335Z | Helper1:
24: tokio::runtime::scheduler::multi_thread::worker::Context::run
2024-09-24T06:55:58.335Z | Helper1:
25: tokio::runtime::context::runtime::enter_runtime
2024-09-24T06:55:58.335Z | Helper1:
26: tokio::runtime::scheduler::multi_thread::worker::run
2024-09-24T06:55:58.335Z | Helper1:
25: tokio::runtime::context::runtime::enter_runtime
2024-09-24T06:55:58.335Z | Helper1:
25: tokio::runtime::context::runtime::enter_runtime
2024-09-24T06:55:58.335Z | Helper1:
26: tokio::runtime::scheduler::multi_thread::worker::run
2024-09-24T06:55:58.335Z | Helper1:
27: tokio::runtime::task::core::Core<T,S>::poll
2024-09-24T06:55:58.335Z | Helper1:
26: tokio::runtime::scheduler::multi_thread::worker::run
2024-09-24T06:55:58.335Z | Helper1:
26: tokio::runtime::scheduler::multi_thread::worker::run
2024-09-24T06:55:58.335Z | Helper1:
27: tokio::runtime::task::core::Core<T,S>::poll
2024-09-24T06:55:58.336Z | Helper1:
28: tokio::runtime::task::harness::Harness<T,S>::poll
2024-09-24T06:55:58.335Z | Helper1:
27: tokio::runtime::task::core::Core<T,S>::poll
2024-09-24T06:55:58.336Z | Helper1:
28: tokio::runtime::task::harness::Harness<T,S>::poll
2024-09-24T06:55:58.335Z | Helper1:
27: tokio::runtime::task::core::Core<T,S>::poll
2024-09-24T06:55:58.336Z | Helper1:
28: tokio::runtime::task::harness::Harness<T,S>::poll
2024-09-24T06:55:58.336Z | Helper1:
29: tokio::runtime::blocking::pool::Inner::run
2024-09-24T06:55:58.336Z | Helper1:
28: tokio::runtime::task::harness::Harness<T,S>::poll
2024-09-24T06:55:58.336Z | Helper1:
29: tokio::runtime::blocking::pool::Inner::run
2024-09-24T06:55:58.336Z | Helper1:
28: tokio::runtime::task::harness::Harness<T,S>::poll
2024-09-24T06:55:58.336Z | Helper1:
29: tokio::runtime::blocking::pool::Inner::run
2024-09-24T06:55:58.336Z | Helper1:
30: std::sys_common::backtrace::__rust_begin_short_backtrace
2024-09-24T06:55:58.336Z | Helper1:
31: core::ops::function::FnOnce::call_once{{vtable.shim}}
2024-09-24T06:55:58.336Z | Helper1:
30: std::sys_common::backtrace::__rust_begin_short_backtrace
2024-09-24T06:55:58.336Z | Helper1:
31: core::ops::function::FnOnce::call_once{{vtable.shim}}
2024-09-24T06:55:58.336Z | Helper1:
32: std::sys::pal::unix::thread::Thread::new::thread_start
2024-09-24T06:55:58.336Z | Helper1:
31: core::ops::function::FnOnce::call_once{{vtable.shim}}
2024-09-24T06:55:58.336Z | Helper1:
32: std::sys::pal::unix::thread::Thread::new::thread_start
2024-09-24T06:55:58.337Z | Helper1:
33: start_thread
2024-09-24T06:55:58.337Z | Helper1:
34: __clone3
2024-09-24T06:55:58.337Z | Helper1:
33: start_thread
2024-09-24T06:55:58.337Z | Helper1:
34: __clone3
2024-09-24T06:55:58.337Z | Helper1:
2024-09-24T06:55:58.337Z | Helper1:
34: __clone3
2024-09-24T06:55:58.337Z | Helper1:
34: __clone3
2024-09-24T06:55:58.337Z | Helper1:
2024-09-24T06:55:58.337Z | Helper1:
thread 'tokio-runtime-worker' panicked at ipa-core/src/protocol/context/batcher.rs:251:9:
2024-09-24T06:55:58.337Z | Helper1:
2024-09-24T06:55:58.337Z | Helper1:
2024-09-24T06:55:58.337Z | Helper1:
thread 'tokio-runtime-worker' panicked at ipa-core/src/protocol/context/batcher.rs:251:9:
2024-09-24T06:55:58.337Z | Helper1:
2024-09-24T06:55:58.337Z | Helper1:
thread 'tokio-runtime-worker' panicked at ipa-core/src/protocol/context/batcher.rs:251:9:
2024-09-24T06:55:58.337Z | Helper1:
assertion failed: self.batches.len() <= 1
2024-09-24T06:55:58.337Z | Helper1:
thread 'tokio-runtime-worker' panicked at ipa-core/src/protocol/context/batcher.rs:251:9:
2024-09-24T06:55:58.337Z | Helper1:
assertion failed: self.batches.len() <= 1
2024-09-24T06:55:58.337Z | Helper1:
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
2024-09-24T06:55:58.338Z | Helper1:
2024-09-24T06:55:58.329439Z INFO oprf_ipa_query{sz=4000000}:attribute_cap_aggregate: ipa_core::protocol::ipa_prf::prf_sharding: close time.busy=193s time.idle=440s

@akoshelev
Copy link
Collaborator Author

^ that was a false alarm, I was running draft on main.

With some other fixes, we are getting to 25M but there is another error

2024-09-25T01:04:01.190427 - 2024-09-25T01:04:01.190370Z  INFO oprf_ipa_query{sz=25000000}:compute_prf_for_inputs: ipa_core::protocol::context::batcher: is_ready_for_validation: batch 190 is ready for validation
2024-09-25T01:04:11.257442 - query_id='6d647286-1f06-4d9f-9502-ad751c91d4e3' running. tailing log file.
2024-09-25T01:04:13.252435 - 2024-09-25T01:04:13.252364Z  INFO oprf_ipa_query{sz=25000000}:compute_prf_for_inputs: ipa_core::protocol::context::batcher: is_ready_for_validation: batch 191 is ready for validation
2024-09-25T01:04:13.292487 - 2024-09-25T01:04:13.292422Z  INFO oprf_ipa_query{sz=25000000}:compute_prf_for_inputs: ipa_core::protocol::context::dzkp_validator: validating 52428800 multiplications
2024-09-25T01:04:21.762627 - 2024-09-25T01:04:21.762551Z  INFO oprf_ipa_query{sz=25000000}:compute_prf_for_inputs: ipa_core::protocol::context::batcher: is_ready_for_validation: batch 192 is ready for validation
2024-09-25T01:04:21.765495 - 2024-09-25T01:04:21.762674Z ERROR oprf_ipa_query{sz=25000000}:compute_prf_for_inputs: ipa_core::error: ThreadId(42) "query_runtime" panicked at ipa-core/src/protocol/context/step.rs:33:10:
2024-09-25T01:04:21.790548 - Step index 192 out of bounds for DzkpBatchStep with count 192. Consider using bounds-checked step constructors.
2024-09-25T01:04:21.791032 - stack trace:

2024-09-25T01:04:21.790548 - Step index 192 out of bounds for DzkpBatchStep with count 192. Consider using bounds-checked step constructors.

this seems like we underestimated the number of steps we need to run malicious validation

@andyleiserson
Copy link
Collaborator

You can increase that back to 256, or even higher. This is #1269. I don't think it's so much that we underestimated the number of steps, it was just a convenient shortcut at the time.

@andyleiserson andyleiserson mentioned this issue Sep 27, 2024
Merged
2 tasks
@andyleiserson andyleiserson self-assigned this Oct 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@andyleiserson andyleiserson

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@akoshelev @andyleiserson