feat(oppool): Return reorged ops to the mempool #276

dphilipson · 2023-07-31T10:03:02Z

Previously, ops could be lost forever if they were mined into a block,
but then that block got reorged away. Now, we detect reorgs and return
any ops contained in them to our mempool.

This requires some involved changes:

We replace the entire events mod and all its listeners. Instead, we
introduce a new type, Chain, which represents our current knowledge
of what blocks make up the blockchain.
We watch for the latest block hash to change. When it does, we read
backwards from the latest block until we connect back to our known
blocks, which lets us see any blocks that were replaced as well.
This process produces a ChainUpdate event, which is sent to the op
pool (replacing NewBlockEvent). This may cause the op pool to not
only remove mined ops but also restore unmined ops.
In order for the op pool to do so, it remembers mined ops for a time
rather than deleting them fully. We add new methods to the op pool for
restoring unmined blocks to make this possible.

[Closes/Fixes] # #87

dphilipson · 2023-07-31T10:04:04Z

I seriously need to write some tests for this, but wanted to get a version up first for discussion and see if anything needs changing (especially for the upcoming op pool server refactor) before I go ham on the mocks.

src/cli/pool.rs

src/common/retry.rs

src/op_pool/mempool/uo_pool.rs

dancoombs · 2023-07-31T15:22:00Z

src/op_pool/mempool/pool.rs

+            self.mined_at_block_number_by_hash.remove(&hash);
+            self.mined_hashes_with_block_numbers.remove(&(bn, hash));


The goal of the op_pool_size_bytes metric is to track memory usage of the pool. We should probably take a look at adding new metrics/updating the old ones to handle these mined operations

src/op_pool/mempool/pool.rs

src/op_pool/chain.rs

dancoombs

One last comment + some testing and this looks good to me! 🚀

src/op_pool/mempool/uo_pool.rs

Previously, ops could be lost forever if they were mined into a block, but then that block got reorged away. Now, we detect reorgs and return any ops contained in them to our mempool. This requires some involved changes: * We replace the entire `events` mod and all its listeners. Instead, we introduce a new type, `Chain`, which represents our current knowledge of what blocks make up the blockchain. * We watch for the latest block hash to change. When it does, we read backwards from the latest block until we connect back to our known blocks, which lets us see any blocks that were replaced as well. * This process produces a `ChainUpdate` event, which is sent to the op pool (replacing `NewBlockEvent`). This may cause the op pool to not only remove mined ops but also restore unmined ops. * In order for the op pool to do so, it remembers mined ops for a time rather than deleting them fully. We add new methods to the op pool for restoring unmined blocks to make this possible.

dphilipson requested a review from dancoombs July 31, 2023 10:03

dphilipson force-pushed the dp/reorgs branch from b81df53 to b37127d Compare July 31, 2023 10:07