Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Logs are not sent if the parser configuration parameter Time_Format is used #128

Open
amosher-github opened this issue Apr 30, 2023 · 0 comments

Comments

@amosher-github
Copy link

Windows 2016
FB version 2.0.9
NR output plugin v1.14.2

It appears that the output plugin does not accept logs that have been parsed with Time_Format.

I have logs (logs from Fluent Bit itself, actually) and have created a parser for them. However if I use Time_Format the logs do not appear in NR. If I comment out the Time_Format parameter, logs appear.

[PARSER]
    # https://rubular.com/r/Ia6N216DDOpL30
    Name fblog
    Format regex
    Regex ^\[(?<time>[^ ]* [^ ]*)\] \[(?<level>[^\]]*)\] \[(?<process>.*?)\] (?<message>[^\]].*)\]?$
    Time_Key time
    Time_Keep On
    Time_Offset -0600
    Time_Format %Y/%m/%d%t%T
[INPUT]
    Name tail
    Tag log.fb
    Path C:\Users\Admin\Documents\fluentbit.log
    Refresh_Interval 1
    Parser fblog
    DB C:\Program Files\fluent-bit\bin\fblog.sqlite

 

To ensure that the parser was actually working I added an [OUTPUT] block to send the parsed logs to a file:

[OUTPUT]
    Name file
    Match *
    Path  C:\Users\Admin\Documents\
    File fblog_after_parse.log
    Format template
    Template {time} Log level: {level} Process: {process} Message: {message}

This worked fine:

1682835222.529526 Log level: debug Process: task Message: created task=0000028179C75510 id=0 OK
1682835222.529534 Log level: debug Process: output:file:file.1 Message: task_id=0 assigned to thread #0
1682835222.529542 Log level: debug Process: out flush Message: cb_destroy coro_id=58
1682835222.529550 Log level: debug Process: out flush Message: cb_destroy coro_id=58
1682835222.529566 Log level: debug Process: task Message: destroy task=0000028179C75510 (task_id=0)
1682835222.529575 Log level: debug Process: input:tail:tail.0 Message: 0 new files found on path 'C:\Users\Admin\Documents\fluentbit.log'
1682835222.793225 Log level: debug Process: input chunk Message: update output instances with new chunk size diff=1059
1682835223.028694 Log level: debug Process: input chunk Message: update output instances with new chunk size diff=167

 

There are no errors in from FB's own error log and the parsed logs go to a file output just fine, but there are no NrIntegrationErrors either so I don't think the output plugin is even sending them at all.

 

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant