Flow Public Key Indexer

A observer service for indexing flow Accounts public keys and REST service that serves up public key data.

Run Parameters

KEYIDX_LOGLEVEL default: "info"
Log Level: Takes string log level value for zerolog, "debug", "info", ...

KEYIDX_PORT default: "8080"
Port: The port the REST service is hosted on

KEYIDX_FLOWURL1 default: "access.mainnet.nodes.onflow.org:9000"
Flow Url: Access node endpoint blockchain data is pulled from, needs to match up with Chain Id. Up to 4 access nodes can be provided, only one is required. The access nodes are cycled through each request to get public key data.

KEYIDX_FLOWURL2 default: none
Flow Url: Access node endpoint

KEYIDX_FLOWURL3 default: none
Flow Url: Access node endpoint

KEYIDX_FLOWURL4 default: none
Flow Url: Access node endpoint

KEYIDX_CHAINID default: "flow-mainnet"
Chain Id: target blockchain, valid values are "flow-testnet" and "flow-mainnet". Needs to match up with Flow Url

KEYIDX_MAXACCTKEYS default: 1000
Max Acct Keys: maximum number of keys to index per account, accounts that have more public keys that exceed max do not get indexed. Accounts are logged out that exceed the max

KEYIDX_BATCHSIZE default: 50000
Batch Size: max number of accounts in a batch sent to cadence script that access node executes. Cadence script can exceed execution if accounts have a lot of keys

KEYIDX_IGNOREZEROWEIGHT default: true
Ignore Zero Weight: tells the cadence script to ignore public keys with zero weight. These keys will not be indexed

KEYIDX_IGNOREREVOKED default: false
Ignore Revoked: tells the cadence script to ignore public keys that have been revoked. These keys will not be indexed

KEYIDX_WAITNUMBLOCKS default: 200
Wait Num Blocks: number of blocks to wait before running an incremental data load

KEYIDX_BLOCKPOLINTERVALSEC default: 180
Block Pol Interval Sec: number of seconds to wait before checking current block height to determine to run an incremental data load

KEYIDX_SYNCDATAPOLINTERVALMIN default: 1
Sync Data Polling Interval: number of minutes to wait between sync data operations

KEYIDX_SYNCDATASTARTINDEX default: 30000000
Sync Data Start Index: starting block height for sync operations

KEYIDX_MAXBLOCKRANGE default: 600
Max Block Range: number of blocks that will trigger a bulk load if services falls behind

KEYIDX_FETCHSLOWDOWNMS default: 500
Fetch Slowdown: milliseconds to wait between fetch operations

KEYIDX_PURGEONSTART default: false
Purge on Start: When changing the data structure or want to clear the database and start from scratch change this variable to true

KEYIDX_ENABLESYNCDATA default: true
Enable Sync Data: Run this service as a sync service. It's possible to run this service only as rest service

KEYIDX_ENABLEINCREMENTAL default: true
Enable Incremental: Enable incremental updates of the database

PostgreSQL configurations

KEYIDX_POSTGRESQLHOST default: "localhost" KEYIDX_POSTGRESQLPORT default: 5432 KEYIDX_POSTGRESQLUSERNAME default: "postgres" KEYIDX_POSTGRESQLPASSWORD not required, no default KEYIDX_POSTGRESQLDATABASE default: "keyindexer" KEYIDX_POSTGRESQLSSL default: true KEYIDX_POSTGRESQLLOGQUERIES default: false KEYIDX_POSTGRESQLSETLOGGER default: false KEYIDX_POSTGRESQLRETRYNUMTIMES default: 30 KEYIDX_POSTGRESQLRETRYSLEEPTIME default: "1s" KEYIDX_POSTGRESQLPOOLSIZE default: 1 KEYIDX_POSTGRESLOGGERPREFIX default: "keyindexer" KEYIDX_POSTGRESPROMETHEUSSUBSYSTEM default: "keyindexer"

Re-indexing Accounts

The service supports re-indexing of specific accounts by adding them to the addressprocessing table. This feature is useful when you need to:

• Update account information that might have changed
• Re-process accounts that may have had errors during initial indexing
• Force a refresh of specific account data

How to Re-index Accounts

1. Add addresses to the addressprocessing table:

INSERT INTO addressprocessing (account) 
VALUES ('0x1234...'), ('0x5678...')
ON CONFLICT (account) DO NOTHING;

2. The service will automatically:
   • Pick up these addresses during the next bulk processing cycle
   • Re-fetch their public key information
   • Update the database with any changes
   • Remove the addresses from the addressprocessing table once processed

Processing Behavior

• Addresses in the addressprocessing table are processed in batches (defined by KEYIDX_BATCHSIZE)
• Duplicate addresses are automatically ignored (ON CONFLICT DO NOTHING)
• Processing occurs during the bulk load cycle (controlled by KEYIDX_SYNCDATAPOLINTERVALMIN)
• After successful processing, addresses are automatically removed from the addressprocessing table

Related Configuration Parameters

• KEYIDX_BATCHSIZE default: 50000
  Controls how many addresses are processed in each batch
  
• KEYIDX_SYNCDATAPOLINTERVALMIN default: 1
  Determines how frequently the service checks for new addresses to process
  

How to Run

Since this is a golang service there are many ways to run it. Below are two ways to run this service

Command line

go run .

Docker

Configuration: Run docker in default 10 gig memory size. Reducing the running memory size reduces performance, the lowest is 6 gig, bulk sync and public key query responses are reasonable compared to running with more memory.
Create a docker container
docker build -t key-indexer .

need to configure to use postgresql This service stores public key data and needs persistent storage in postgresql
Run the docker and map the rest service port
Notice that environmental variables can be passed in. See variables above
docker run -p 8888:8080 --env KEYIDX_POSTGRESQLHOST=localhost
To see the logs of the container, get the container id
docker container ls
View the containers logs
docker logs <container id>

REST service

Endpoints

• GET /key/{public key}

note: public key is in base64 format serves up json object

{
    "publicKey": string,  // public key string in base64
    "accounts": [
        {
            "address": string,    // Flow account address
            "keyId": int,        // Key index in the account
            "weight": int,       // Key weight for signing
            "sigAlgo": int,      // Signing algorithm identifier
            "hashAlgo": int,     // Hashing algorithm identifier
            "signing": string,   // Human-readable signing algorithm name
            "hashing": string    // Human-readable hashing algorithm name
        }
    ]
}

sigAlgo - signing: 1 - ECDSA_P256, 2 - ECDSA_secp256k1, 3 - ECDSA_P384, 4 - ECDSA_P521, 5 - ECDSA_secp256r1

hashAlgo - hashing: 1 - SHA2_256, 2 - SHA3_256, 3 - SHA2_384, 4 - SHA3_384, 5 - SHA2_512, 6 - SHA3_512

• GET /status

note: this endpoint gives ability to see if the server is active and updating

{
    "Count": int,           // Number of unique public keys indexed
    "LoadedToBlock": int,   // Last processed block height
    "CurrentBlock": int     // Current block height on the Flow network
}