Skip to content

Tools for exporting blockchain data to analytical databases

License

Notifications You must be signed in to change notification settings

coinmetrics/haskell-tools

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Haskell-based CoinMetrics.io tools

These tools are used by CoinMetrics.io team for exporting data from blockchains into analytical databases and monitoring full nodes synchronization state.

Utilities

  • coinmetrics-export - utility for exporting data from blockchains in formats suitable for inserting into analytics databases (SQL, Avro).
  • coinmetrics-monitor - utility for monitoring blockchain nodes and providing Prometheus-compatible metrics.

Status

The project is being used in production at Coin Metrics, but many things are in flux or fragile, and docs may be outdated. Command line interface is more or less stable but may change. Please use with caution.

Supported blockchains:

Binaries

There're no stable releases yet. All binaries are "bleeding edge".

One easy way to run the tools is to use docker.

Pull the latest version:

docker pull coinmetrics/haskell-tools

Run e.g. coinmetrics-export tool:

docker run -it --rm --net host coinmetrics/haskell-tools coinmetrics-export <arguments>

Building from source

Building with Nix is recommended, because dependencies are fetched and built automatically.

With Nix

nix build -Lf ./release.nix bins

With Stack + Nix

stack build --nix

With Stack

stack build

Required dependencies: zlib, libpq.

Stellar export also requires xdrpp and Stellar XDR headers built with XDR compiler, which may be hard to build manually. By default Stellar support is enabled, so build will fail without them. You can disable Stellar support if you don't need it: stack build --flag coinmetrics-all-blockchains:-stellar. If you do need it, please use one of the Nix build methods above (with Stack or without).

The code is only tested on Linux (but maybe works on other OSes too).