Store witness #174
Store witness #174
Conversation
|
We require contributors/corporates @anshalshukla to read our Contributor License Agreement, please check the Individual CLA document/Corporate CLA document |
eth/stagedsync/stage_blockhashes.go
Outdated
| @@ -61,6 +61,11 @@ func SpawnBlockHashStage(s *StageState, tx kv.RwTx, cfg BlockHashesCfg, ctx cont | |||
| return nil | |||
| } | |||
|
|
|||
| // Move 100 block at a time, required for witness generation stage | |||
| if headNumber-s.BlockNumber > 100 { | |||
| headNumber = s.BlockNumber + 100 | |||
There was a problem hiding this comment.
It would be great if you can explain why we need to move 100 block at a time for witness stage. Also, is it possible not to use a hard coded number?
zk/hermez_db/db.go
Outdated
| return nil | ||
| } | ||
|
|
||
| const chunkSize = 100000 // 100KB |
| { | ||
| ID: stages2.Witness, | ||
| Description: "Generates and stores witness of batches", | ||
| Forward: func(firstCycle bool, badBlockUnwind bool, s *stages.StageState, u stages.Unwinder, tx kv.RwTx, quiet bool) error { | ||
| return SpawnWitnessStage(s, u, tx, ctx, witnessCfg, firstCycle, quiet) | ||
| }, | ||
| Unwind: func(firstCycle bool, u *stages.UnwindState, s *stages.StageState, tx kv.RwTx) error { | ||
| return UnwindWitnessStage(u, s, tx, ctx, witnessCfg, firstCycle) | ||
| }, | ||
| Prune: func(firstCycle bool, p *stages.PruneState, tx kv.RwTx) error { | ||
| return PruneWitnessStage(p, tx, witnessCfg, ctx, firstCycle) | ||
| }, | ||
| }, |
There was a problem hiding this comment.
Can we add a flag to cli which users can enable/disable this stage?
zk/stages/stages.go
Outdated
| { | ||
| // First cycle 1-100 | ||
| // First batch - 1-104 | ||
| // Second cycle - witness generation | ||
| ID: stages2.Witness, | ||
| Description: "Generates and stores witness of batches", | ||
| Forward: func(firstCycle bool, badBlockUnwind bool, s *stages.StageState, u stages.Unwinder, tx kv.RwTx, quiet bool) error { | ||
| return SpawnWitnessStage(s, u, tx, ctx, witnessCfg, firstCycle, quiet) | ||
| }, | ||
| Unwind: func(firstCycle bool, u *stages.UnwindState, s *stages.StageState, tx kv.RwTx) error { | ||
| return UnwindWitnessStage(u, s, tx, ctx, witnessCfg, firstCycle) | ||
| }, | ||
| Prune: func(firstCycle bool, p *stages.PruneState, tx kv.RwTx) error { | ||
| return PruneWitnessStage(p, tx, witnessCfg, ctx, firstCycle) | ||
| }, | ||
| }, |
|
We require contributors/corporates @anshalshukla to read our Contributor License Agreement, please check the Individual CLA document/Corporate CLA document |
4 similar comments
|
We require contributors/corporates @anshalshukla to read our Contributor License Agreement, please check the Individual CLA document/Corporate CLA document |
|
We require contributors/corporates @anshalshukla to read our Contributor License Agreement, please check the Individual CLA document/Corporate CLA document |
|
We require contributors/corporates @anshalshukla to read our Contributor License Agreement, please check the Individual CLA document/Corporate CLA document |
|
We require contributors/corporates @anshalshukla to read our Contributor License Agreement, please check the Individual CLA document/Corporate CLA document |
zk/hermez_db/db.go
Outdated
| } | ||
|
|
||
| // Removes `_chuck_ + chunk_id` that we add while storing it in db | ||
| batch := key[0:9] |
There was a problem hiding this comment.
It wil be you can refer(https://github.com/0xPolygonHermez/cdk-erigon/blob/zkevm/zk/hermez_db/utils.go#L18), also I found out there is a function already in the same file which convert bytesToUint64 so I can remove the function, but the logic there is pretty much the same.
|
We require contributors/corporates @anshalshukla to read our Contributor License Agreement, please check the Individual CLA document/Corporate CLA document |
cmd/rpcdaemon/commands/zkevm_api.go
Outdated
| startBlock := rpc.BlockNumberOrHashWithNumber(rpc.BlockNumber(blocks[len(blocks)-1])) | ||
| return api.getBlockRangeWitness(ctx, api.db, startBlock, endBlock, false) | ||
| hermezDb := hermez_db.NewHermezDbReader(tx) | ||
| return hermezDb.GetWitnessByBatchNo(batchNumber) |
There was a problem hiding this comment.
Should it fall back to witness calculation if the witness is not found in db, e.g. when witness for a batch is pruned?
zk/stages/stage_witness.go
Outdated
| // Only storeNBatches witness in db | ||
| // Won't enforce a hard count as batches aren't contiguous | ||
| // Cannot use cursor count to get the exact number as batch | ||
| // witnesses are stored in chunks (number will vary from batch to batch) | ||
| if (b - oldestBatch + 1) > cfg.storeNBatches { | ||
| hermezDb.DeleteWitnessByBatchNo(oldestBatch) | ||
| } |
There was a problem hiding this comment.
When using batch number, we can't control the exact max number of witnesses stored in db, because batch number is not contiguous as you mentioned. If we use block number instead, we will have a more accurate control of how many blocks' witnesses we want to keep in the db. Basically, changing storeNBatches to storeNBlocks, and check if the batch number of the lower bound block (currentBlockN - storeNBatches - 1) is stored in db. If so, delete the batch.
|
We require contributors/corporates @anshalshukla to read our Contributor License Agreement, please check the Individual CLA document/Corporate CLA document |
|
LGTM, thanks! Could you rebase the branch? |
|
We require contributors/corporates @anshalshukla to read our Contributor License Agreement, please check the Individual CLA document/Corporate CLA document |
anshalshukla
force-pushed
the
store-witness
branch
from
825d846
to
e1afe6c
Compare
|
We require contributors/corporates @anshalshukla to read our Contributor License Agreement, please check the Individual CLA document/Corporate CLA document |
anshalshukla
force-pushed
the
store-witness
branch
from
e1afe6c
to
7f0c9f2
Compare
|
We require contributors/corporates @anshalshukla to read our Contributor License Agreement, please check the Individual CLA document/Corporate CLA document |
|
|
@anshalshukla great contribution, please could you comment something like 'I accept the terms of the CLA' and I will summon the CLA bot :D |
|
Ah, I thought I had to add it on cla repo. I had create a separate PR for that 😆 |
|
I accept the terms of the CLA |
No description provided.