Chevrotain error: Unable to use "first char" lexer optimizations

[bankyadam]

	Failed parsing: < /('''|""")[\n\s\S]+\1|(?:((?<![\\])['"])((?:.(?!(?<![\\])\2))*.?)\2)/ >
	Using the regexp-to-ast library version: 0.5.0
	Please open an issue at: https://github.com/bd82/regexp-to-ast/issues

To help to understand the regexp, here are my test strings that this expression accepts:

"string"

"string with ' in it"

"string \"quoted\" inside"

'string'

'string with " in it'

'string \'quoted\' inside'

'''multi
line
string'''

'''can contain (') in it'''

'''can contain (") in it'''

"""can contain (") in it"""

"""can contain (') in it"""

[bd82]

Hello @bankyadam and thanks for reporting this.

I am currently investigating deprecating the regexp-to-ast package and replacing its usage in Chevrotain
with another regexp parser.

• Investigate using regexpp Instead of regexp-to-ast Chevrotain/chevrotain#777

This means this issue will have a low priority as it may become redundant.

What this means for the Chevrotain Parser is that some performance optimizations will not take effect for your lexer. You may be able to work-around this by specifying additional metadata on your Token object.
See: https://sap.github.io/chevrotain/docs/guide/performance.html#ensuring-lexer-optimizations

[masukomi]

this is still a problem in 2024... :/

      	Failed parsing: < /((?<!\\)'''(.|\n)*?(?<!\\)''')|((?<!\\)"""(.|\n)*?(?<!\\)""")|((?<!\\)".*?(?<!\\)")|((?<!\\)'.*?(?<!\\)')/y >
      	Using the regexp-to-ast library version: 0.5.0
      	Please open an issue at: https://github.com/bd82/regexp-to-ast/issues

[bd82]

Hello @masukomi

This library was moved to https://github.com/Chevrotain/chevrotain/tree/master/packages/regexp-to-ast
This repo is no longer active...

The error message you are printing shows you are using an old version of Chevrotain.
You should try to reproduce the issue with the newest version of the library and/or chevrotain.
If it does reproduce you should then attempt to a minimal regexp pattern which causes the issue.
It likely indicates an unsupported reg-exp syntax feature...

[bd82]

If the issue still reproduces open an issue at https://github.com/Chevrotain/chevrotain/issues