🐛 "Timeout waiting for endorser to accept the revocations request"

[ff137]

Our end-to-end tests for proving revoked credentials will every now and then fail with:

ERROR app/tests/e2e/verifier/test_proof_revoked_credential.py::test_proof_revoked_credential[clean-clean-clean-clean-clean-clean-v2-default] - fastapi.exceptions.HTTPException: 504: {"detail":"Timeout waiting for endorser to accept the revocations request."}

Additionally, when we attempt to assert that a revoked credential returns valid: False in a proof request flow, if the proof is done too soon after revocation is submitted, it may not reflect on the ledger or tails-server in time, resulting in:

FAILED app/tests/e2e/verifier/test_proof_revoked_credential.py::test_proof_revoked_credential[clean-clean-clean-clean-clean-clean-v2-auto_publish_true] - assert True is False

So, the objective should be to make these functions and tests more resilient to timeouts or race conditions, to reduce spurious test errors, but also to be able to more surely guarantee desired functionality to clients.

[ff137]

https://github.com/didx-xyz/aries-cloudapi-python/actions/runs/10288809447/job/28475148939?pr=961

[ff137]

https://github.com/didx-xyz/aries-cloudapi-python/actions/runs/10389247224/job/28766894748

[ff137]

Sample relevant ledger-nodes warning (client error):

2024-09-10 16:12:03,197|WARNING|ordering_service.py|Node1:0 encountered exception InvalidClientRequest('Eo6nJ2YL8eH36ZqmH147VB', 1725984722961626822, 'The current accumulator value: 21 11C560AD909E6A0DEDDA821CF1CA42697D85708EA0B6054EAD175E8096D9B3B31 21 127EE85E44FE620AB0A255D1AC7820A70470314AD0D3DA2451322CB1FB50F9BA9 6 90BB99FBF88FD7BB6BA70369E84828DDFE8540BB9950E1E7C732BF6ABFF80AEB 4 20DCE705EDE3D9823BF614E582AB4A84A7F72178E75FB6EC9F87F43730283CD5 6 88623B62DC3980619ACCA6E947C129106C52FC8245D89BD6857931477946BEE8 4 36D9054E47BA8C09BDCF9E4D27E377920C2EA54EA07822C16C6BA8D21427A47A must be equal to the last accumulator value: 21 140ABB9634CB019512BFD1F2D2778D1E42E7024B56C35CBFDF26BE5076AD786F9 21 1300F18CE36A84D8CFFA168E9D045B9BDD52A7E29346BE752A7FABB4DF6B699AD 6 7DC9C870350F7D99304B207875CDCC8F0B8F0EC7BA3B578C808FBBF70E61437F 4 24252B10FB1999EF05AE4FEDA774D05A4B2C6DEB9EA5745F07FB8F92D5078A56 6 5BCE1C2F1943BEFDB1A8C75F1E76862AE4F8F49BE8A9AB49E97F66E4CA708F06 4 330BE3B86F42B32F0E0A4C87BB066DD0D173A5A51386D463D7A277B38365B58D in transaction') while processing Request: {'reqId': 1725984722961626822, 'operation': {'revocDefType': 'CL_ACCUM', 'revocRegDefId': 'Eo6nJ2YL8eH36ZqmH147VB:4:Eo6nJ2YL8eH36ZqmH147VB:3:CL:261:tag:CL_ACCUM:9cd52964-d399-4bc1-85c2-71e87b0ce317', 'type': '114', 'value': {'accum': '21 12F68929FA8683A5038294A0E6913A15A9B4D1C2E107ABE297E832F8DD69395F9 21 131131BE5C79D9CC35D765C721EEC336E622C59DD858B1185B7444D22B5A2A821 6 6F28CC06AD893E2922D61825AFAB4BCF16C657490B4BC1EF9A5374BB159AD89B 4 22B825C8A96FBFAF711D542AACF9B6C6F30E8A30EBA41418E978769C472CF159 6 817EE5B4A70B2F3DFD2C1F42C0EB54CEDC8A4B037158DBDE6FE8C33EB6F6DF93 4 21DABD221C974EFBA349A0C62B42B316E061AC84635EE7FCF9E8199FBD2C540C', 'prevAccum': '21 11F1828E6F8B5267CD55B910541822155319BB9D19A56E72A170188A084179C95 21 131C1CD605630C7158EABA328A4DDE4772AD5479D83CA34E38B40E653E35A4A6A 6 7BBF93B4F6ABF0D13E73BC1C6ED7F64DF6D1B0E3160CBFCE266A6BE008C3CA77 4 2FA52DF703ED761E33E72A9B00EB96BB33EC254964098F44748DA0C65809835D 6 886BE58E9039C508D5B939142572066F7D034A86CDCE8081E4D2B2264A7E39C6 4 1CE7410D622B27C7B2BD2F91789F74F96FA66C43B9BDEEE7D00A80FE5C5FFC71', 'revoked': [149, 177, 109, 181, 182, 190, 191, 121, 101, 185, 192, 199, 122, 197, 130, 166, 115, 137, 147, 150, 178, 193, 194, 148, 189, 140, 152, 156, 116, 105, 183, 102, 125, 163, 196, 132, 179, 157, 184, 165, 154, 114, 133, 135, 153, 187, 131, 103, 134, 172, 108, 173, 169, 144, 200, 170, 167, 198, 111, 110, 117, 151, 186, 142, 128, 195, 139, 106, 138, 155, 175, 146, 188, 159, 127, 171, 107, 126, 160, 136, 129, 143, 161, 162, 118, 123, 124, 164, 141, 119, 180, 120, 158, 104, 112, 145, 168, 174, 176, 113]}}, 'identifier': 'Eo6nJ2YL8eH36ZqmH147VB', 'signatures': {'947ZS9bdgWMY74JCbGFZhC': '29nwyVtArqGx76DZx8qpMasEcDA4mKn9aT5A8XTLDNijY3Tu2gTZjFdq4csMaTajTTui6dxEPBpmv24f2PSTewVE', 'Eo6nJ2YL8eH36ZqmH147VB': 'x8Cb5haawFdq6jfEVQ2bV4QucWWYzAuEKj6czYtvPGiH5h34xrn9ef8vTVGFCVsC6YY6WM6tEb8bn7u2SJsDiyw'}, 'protocolVersion': 2, 'taaAcceptance': {'mechanism': 'service_agreement', 'taaDigest': '0be4d87dec17a7901cb8ba8bb4239ee34d4f6e08906f3dad81d1d052dccc078f', 'time': 1725926400}, 'endorser': '947ZS9bdgWMY74JCbGFZhC'}, will reject

Sample error in multitenant-agent:

indy_vdr.error.VdrError: Request failed: client request invalid: InvalidClientRequest('Eo6nJ2YL8eH36ZqmH147VB', 1725984722961626822, 'The current accumulator value: 21 11C560AD909E6A0DEDDA821CF1CA42697D85708EA0B6054EAD175E8096D9B3B31 21 127EE85E44FE620AB0A255D1AC7820A70470314AD0D3DA2451322CB1FB50F9BA9 6 90BB99FBF88FD7BB6BA70369E84828DDFE8540BB9950E1E7C732BF6ABFF80AEB 4 20DCE705EDE3D9823BF614E582AB4A84A7F72178E75FB6EC9F87F43730283CD5 6 88623B62DC3980619ACCA6E947C129106C52FC8245D89BD6857931477946BEE8 4 36D9054E47BA8C09BDCF9E4D27E377920C2EA54EA07822C16C6BA8D21427A47A must be equal to the last accumulator value: 21 140ABB9634CB019512BFD1F2D2778D1E42E7024B56C35CBFDF26BE5076AD786F9 21 1300F18CE36A84D8CFFA168E9D045B9BDD52A7E29346BE752A7FABB4DF6B699AD 6 7DC9C870350F7D99304B207875CDCC8F0B8F0EC7BA3B578C808FBBF70E61437F 4 24252B10FB1999EF05AE4FEDA774D05A4B2C6DEB9EA5745F07FB8F92D5078A56 6 5BCE1C2F1943BEFDB1A8C75F1E76862AE4F8F49BE8A9AB49E97F66E4CA708F06 4 330BE3B86F42B32F0E0A4C87BB066DD0D173A5A51386D463D7A277B38365B58D in transaction')