Remove timestamp from Fluent-bit logs.

[vijaynauti]

Hi All.
My logs under the OpenSearch is showing like below. Trying to remove he timestamp which is already appended to logs but not finding proper document for the same. Also trying to achieve the same with LUA,

2023/01/23 16:13:25 [error] 30#30: *3 open() "/usr/share/nginx/html/phpMyAdmin-5.3.0-all-languages/index.php" failed (2: No such file or directory), client: 66.94.123.118, server: localhost, request: "GET /phpMyAdmin-5.3.0-all-languages/index.php?lang=en HTTP/1.1", host:
@PettitWesley

[patrick-stephens]

I think the issue is you're not parsing the timestamp from the log message so it is being included - typically you would extract this to use as the timestamp for the log record rather than the local Fluent Bit time it is seen.

You can parse the timestamp so it is used then as the Fluent Bit timestamp and no longer included in the message or you could have a LUA filter that removes that first timestamp section. You could also have a parser filter also skip over it in a regex.