Found an online challenge for Messari a Web3 crypto research & data company and thought it would be a good idea to attempt doing it to practice Go programming language which I am currently learning.
The purpose of the challenge is to write a tool that parses trade data as it comes in and compute various aggregate metrics from the provided data, completing a set of ten million trades in as little time as possible.
A binary that writes ten million trade objects as JSON to stdout (stdoutinator) is provided. The tool should accept input from stdin so output from the provided binary can be piped into it using a terminal window.
This is actually my second Go script so the language best practices and conventions may not be top notch here.
- Challenge
- Slices
- Variable Weighted Average Price
- bufio Reader vs Scanner benchmarking
- Reading line by lines in go
- Go perfbook
Make sure you have Go installed, more info here.
From a terminal where the main.go file is located just run go build to build the market-data-aggregator executable
To pipe stdoutinator's stdout to market-data-aggregator's stdin, in MacOS use the following command. Note that the terminal must be open in the location that both executables are located.
exec ./stdoutinator | ./market-data-aggregator
You should see a terminal output like the one below.
➜ market-data-aggregator git:(master) ✗ exec ./stdoutinator | ./market-data-aggregator
Starting...
Duration of aggregate operation: 20.201481417s
Total trades: 10000001
Total markets: 8831
Writing to file...
A markets.json file will be generated containing all aggregated markets data from all trades.