Wallet Proxy does not return a result

[sderuiter]

Bug Description
Trying to retrieve transactions for a specific date, specifying blockTimeFrom and blockTimeTo. However, the api never returns a result.

Steps to Reproduce
curl -XGET https://wallet-proxy.mainnet.concordium.software/v1/accTransactions/3XSLuJcXg6xEua6iBPnWacc3iWh93yEDMCqX8FbE3RDSbEnT9P?limit=1000&finalizationRewards=n&blockRewards=n&order=d&blockTimeFrom=1644015600&blockTimeTo=1644101999

Expected Result
A response

Actual Result
504 Gateway Time-out.

Versions

• Software Version: ?
• OS Mac OS Montery 12.1
• Browser: terminal

[sderuiter]

To add: when (for other days/accounts) it does return a result, it often takes more than 120 seconds to do so.

[abizjak]

Hi.

Thanks for reporting, but this is a known issue with the current design of the database. There is an ongoing effort to improve the database indices and the structure of the tables to support more efficient filtering, in particular by transaction type and time, but it is low priority compared to other features we have for the next release.

[sderuiter]

Ok thank you. Is there another way to reliably retrieve a list of transactions for a day?

[abizjak]

For a specific day in the past not with the wallet-proxy, unless your account has lots of normal transactions for every day.

If you have access to a node then it'll be quite quick to just query those day's blocks and list. Here is how that would look with the rust SDK (run with a command

list-transactions --from 2022-02-09T00:00:00Z --to 2022-02-10T00:00:00Z --target ... --num 8

where target would be your account address.

use anyhow::Context;
use chrono::Utc;
use clap::AppSettings;
use concordium_rust_sdk::{
    endpoints,
    types::{queries::BlockInfo, AbsoluteBlockHeight, BlockItemSummaryDetails, BlockSummary},
};
use id::types::AccountAddress;
use structopt::StructOpt;

#[derive(StructOpt)]
struct App {
    #[structopt(
        long = "node",
        help = "GRPC interface of the node.",
        default_value = "http://localhost:10000"
    )]
    endpoint: tonic::transport::Endpoint,
    #[structopt(
        long = "num",
        help = "How many queries to make in parallel.",
        default_value = "1"
    )]
    num:      u64,
    #[structopt(long = "from", help = "Starting time. Defaults to genesis time.")]
    from:     Option<chrono::DateTime<chrono::Utc>>,
    #[structopt(long = "to", help = "End time. Defaults to infinity.")]
    to:       Option<chrono::DateTime<Utc>>,
    #[structopt(
        long = "target",
        help = "Only list transfers that affect this account."
    )]
    target:   AccountAddress,
}

#[tokio::main(flavor = "multi_thread")]
async fn main() -> anyhow::Result<()> {
    let app = {
        let app = App::clap().global_setting(AppSettings::ColoredHelp);
        let matches = app.get_matches();
        App::from_clap(&matches)
    };

    let mut client = endpoints::Client::connect(app.endpoint, "rpcadmin".to_string()).await?;

    let target_addr = &app.target;

    let cs = client.get_consensus_status().await?;

    // Find the block to start at.
    let mut h = if let Some(start_time) = app.from {
        let cb = cs.last_finalized_block;
        let mut bi = client.get_block_info(&cb).await?;
        anyhow::ensure!(
            bi.block_slot_time >= start_time,
            "Last finalized block is not after the requested start time ({})",
            bi.block_slot_time.to_rfc3339()
        );
        let mut end: u64 = bi.block_height.into();
        let mut start = 0;
        while start < end {
            let mid = (start + end) / 2;
            let bh = client
                .get_blocks_at_height(AbsoluteBlockHeight::from(mid).into())
                .await?[0];
            bi = client.get_block_info(&bh).await?;
            if bi.block_slot_time < start_time {
                start = mid + 1;
            } else {
                end = mid;
            }
        }
        start.into()
    } else {
        AbsoluteBlockHeight::from(0u64)
    };
    // Query blocks by increasing height.
    loop {
        let mut handles = Vec::with_capacity(app.num as usize);
        for height in u64::from(h)..u64::from(h) + app.num {
            let cc = client.clone();
            handles.push(tokio::spawn(async move {
                let h: AbsoluteBlockHeight = height.into();
                let mut cc = cc.clone();
                let blocks = cc
                    .get_blocks_at_height(h.into())
                    .await
                    .context("Blocks at height.")?;
                if blocks.len() == 0 {
                    return Ok::<Option<(BlockInfo, Option<BlockSummary>)>, anyhow::Error>(None);
                }
                let bi = cc.get_block_info(&blocks[0]).await.context("Block info.")?;
                if !bi.finalized {
                    return Ok::<_, anyhow::Error>(None);
                }
                if bi.transaction_count != 0 {
                    let summary = cc
                        .get_block_summary(&blocks[0])
                        .await
                        .context("Block summary.")?;
                    Ok(Some((bi, Some(summary))))
                } else {
                    Ok::<_, anyhow::Error>(Some((bi, None)))
                }
            }))
        }
        let mut success = true;
        for res in futures::future::join_all(handles).await {
            if let Some((bi, summary)) = res?? {
                if let Some(end) = app.to.as_ref() {
                    if end <= &bi.block_slot_time {
                        return Ok(());
                    }
                }
                h = h.next();
                if let Some(summary) = summary {
                    for bisummary in summary.transaction_summaries {
                        let affected = bisummary.affected_addresses();
                        let found = affected
                            .iter()
                            .any(|addr| addr.as_ref()[0..29] == target_addr.as_ref()[0..29]);
                        if found {
                            // Only print transactions. If rewards are needed add output of that
                            // here as well.
                            if let BlockItemSummaryDetails::AccountTransaction(at) =
                                bisummary.details
                            {
                                let sender = at.sender;
                                let is_sender =
                                    target_addr.as_ref()[0..29] == sender.as_ref()[0..29];
                                // output hash, and whether it is a sender.
                                println!("{}, {}", bisummary.hash, is_sender);
                            }
                        }
                    }
                }
            } else {
                success = false;
                break;
            }
        }
        if !success {
            // if we failed and end time is not yet here, then wait a bit
            tokio::time::sleep(std::time::Duration::from_millis(500)).await;
        }
    }
}

[abizjak]

Would this solution be acceptable to you (with modifications to suit your precise needs of course)?

[sderuiter]

That's not a concordium-client you're referring to, correct? If not, this solution will not fit for me (zk on Rust...).

[abizjak]

No, it is not concordium-client. I assume your parenthetical remark means you have no knowledge of rust :)

[sderuiter]

Unfortunately...Eager to learn, though, but at this point in time that's not feasible.

[sderuiter]

I'm using this to estimate token inflation through baking rewards and finalization rewards. Is there a way to retrieve the daily token inflation via a more direct route?

[abizjak]

Yes indeed there is.

$ concordium-client raw GetRewardStatus 0fbe99f7d33d45dcee79ec08dd5ffdda79a46c2f0c62fd18aaa3e37153012494
{
    "finalizationRewardAccount": "19",
    "totalEncryptedAmount": "689913982188",
    "totalAmount": "10714743963296850",
    "gasAccount": "3",
    "bakingRewardAccount": "9929776821"
}

You can see the total amount of CCD (expressed in microCCD) at the end of a given block. I believe this is exactly what you need.

[sderuiter]

This looks perfect. Is there a method to retrieve a block at a given time (in my case: block at start of day and block at end of day)?

[abizjak]

No, not directly. The best way to go about this is bisection + GetBlocksAtHeight + GetBlockInfo (look at the blockSlotTime field).

[sderuiter]

Ok, thanks. And in the example block you gave above, should I read this as that for this particular block the baking Reward was 9929 CCD, or am I misinterpreting this?

[abizjak]

No, not exactly. Baking rewards are not paid directly to the baker of the block. Instead they are accrued over an epoch and then distributed at the end of the epoch. The value returned here is the amount accrued since the beginning of an epoch (an epoch is currently one hour).

[sderuiter]

Yes, I had noticed the epoch payment schedules. I'm afraid this requires a level of spelonking + calculation efforts to get this right though. For now I think I'll stick with the slow wallet-proxy...Thanks for your help sofar!

[abizjak]

If you are doing these queries incrementally, you can speed them up by using the from parameter with the id of the last transaction you received. That should speed up.