Stackoverflow when deriving token with block comment regex #400
Labels
Comments
|
After a bit of debugging:
I'll continue to investigate, but any advice would be appreciated! |
|
Hello @KarelPeeters! Thanks for reporting this bug (though I am not sure if this is a bug or a limitation of Logos). Unfortunately, I don't have time to investigate this at the moment. However, your regex seems very complex, and it might be worth trying to simplify it, at least by breaking it down into multiple tokens or using callbacks (this is usually the simplest thing to do when trying to match block comments). |
|
For now, I think handling multiple line comment manually would be better. Here is code snippet from my project. ...
#[token("/*", multiline_comment)]
BlockComment,
...fn multiline_comment(lex: &mut Lexer<TokenType>) -> FilterResult<(), LogosLexError> {
enum State {
ExpectStar,
ExpectSlash,
}
let remainder = lex.remainder();
let (mut state, mut iter) = (State::ExpectStar, remainder.chars());
while let Some(next_char) = iter.next() {
match next_char {
'\n' => {
lex.extras.line += 1;
lex.extras.line_beg = lex.span().end + (remainder.len() - iter.as_str().len());
state = State::ExpectStar;
}
'*' => state = State::ExpectSlash,
'/' if matches!(state, State::ExpectSlash) => {
lex.bump(remainder.len() - iter.as_str().len());
return FilterResult::Skip;
}
_ => state = State::ExpectStar,
}
}
lex.bump(remainder.len());
FilterResult::Error(LogosLexError::IncompleteComment)
} |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The following derive setup causes the build to fail:
The error printed is:
Other tokens (normal literal string, other regular expressions) work fine. I assume this is because somewhere the the derive machinery this specific regex causes infinite recursion.
Note: I got this regex from the LALRPOP book here.
The text was updated successfully, but these errors were encountered: