Handle ledger not found before ETL process has finished

[ckniffen]

Summary

Websocket events are currently proxied to the ETL source. When receiving a ledgerClosed event it is very common to request the new ledger to get all of its transaction details. Sometimes the ledger has not yet been processed by clio yet and lgrNotFound error is returned.

Solution

Solution 1

Do not proxy ledger closed events instead wait to emit them after the new ledger has been ingested and is available to retrieve.

Solution 2

Do not immediately return lgrNotFound if it is within some threshold of the most recently available ledger (maybe 1 or 2) and return once it is available. There could also be a reference to ledgers being ingested and it could be checked to see if the response should be delayed.