[build] Fix for #3575

[kaby76]

This is a fix for #3575, in response to antlr/antlr4test-maven-plugin#88 and #3574 (comment).

Tracking what tokens are produced in a parse is just as important as tracking the parse tree. The main problem with the .tree output file trace is that it doesn't print out detailed information about a token.

The change here automatically computes the tokens and places them in a file named after the input file with ".token" appended. Note, the suffix/extension of the file cannot be ".tokens" because that conflicts with the Antlr4 tool generated .tokens files. The .gitignore contains a pattern to ignore .tokens files!

To track the differences in the tokens produced, simply git add the file to the repo. To generate the file from scratch, run _scripts/test.sh or test.ps1. If you check in the token file, you should updated the pom.xml, and make sure to specify a wildcard pattern for the files to parse, e.g., examples/.sys`. If you do not, the old Maven test plugin will try to parse the .token file!*

The changes are:

• Add code to the drivers to output to a .token file.
• Add code to the target-specific test.* to run with -tokens and reset the lexer back to the beginning. Note, this will slow testing down because the input is lexed twice. But, it is a good test for the reset function of the port.